Nabble - Emacs - Bidi

Re: Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-11-27T20:44:09Z

Hello Eli,

Many thanks for your comments. I very much agree with what you say,
namely that good display for HTML/XML (and similar stuff) should be at a
level above the basic bidi support, and that it may be implemented by
somebody else than you.

Nevertheless, I very much look forward to hear from Gregg (and others)
about what they see as the current problems with bidi editing (such as
the issue with carret placement that Gregg mentioned).

Regards, Martin.

On 2009/11/27 21:24, Eli Zaretskii wrote:

>> Date: Fri, 27 Nov 2009 20:11:47 +0900
>> From: "Martin J. Dürst"<duerst@...>
>> CC: Gregg Reynolds<gar@...>, emacs-bidi@..., emacs-devel@...
>>
>> On 2009/11/27 1:10, Eli Zaretskii wrote:
>>>> Date: Thu, 26 Nov 2009 08:58:39 -0600
>>>> From: Gregg Reynolds<gar@...>
>>
>>>> It's also a major pain to edit XML in a bidi editor. And I do mean
>>>> major.
>>>
>>> Again, an explicit list of main problems would be useful (although I
>>> deliberately decided not to handle bidirectional editing for meta
>>> documents yet, so this specific class of problems is still a long way
>>> from being dealt with).
>>
>> This is a well-known problem. For some more background and some attempt
>> at a solution, please see:
>>
>> http://www.sw.it.aoyama.ac.jp/2005/pub/IUC28-bidi/ and
>> http://www.sw.it.aoyama.ac.jp/2008/pub/IUC32-bidi/
>
> Thanks.
>
> As I pointed out elsewhere, this class of problems is currently beyond
> the scope of what I'm trying to do in Emacs. That is because the
> display engine basically examines characters one by one, so correct
> display of XML/HTML using just UAX#9 algorithm is impossible, as you
> and others have found out.
>
> I think that support for bidi editing of documents that use markup
> languages should be based on the features that exist as part of the
> markup:‎/‏, the dir= directive etc. Using these, a Lisp
> application should place overlays or maybe text properties on the
> portions of text that need to be reordered. For example, one could
> place an overlay on a portion of the buffer with `display' property
> whose value is the reordered text of that portion (it will have to be
> enclosed in LRO..PDF embedding to avoid the automatic reordering when
> the display string is delivered to the glass). Or we could add a new
> overlay property that would just tell the display engine "reorder this
> part". Then the display engine will DTRT guided by these overlays and
> text properties.
>
> However, please note that the above is not something I thought about
> long enough to be convinced that it is the right solution. The only
> thing I convinced myself in at this point is that bidi support for
> markup languages should not be on the basic display engine level,
> which is what I'm currently hacking. Rather, it should use the basic
> reordering capabilities of the display engine in some way, but do some
> additional work on higher levels to implement the equivalent of the
> UAX#9 ``higher protocols''.
>
>> In a discussion with Ken'ichi Handa and Naoto Takahashi in 2005, they
>> pointed out that using bidi marks in overlays should make it possible in
>> bidi emacs to address this problem. Unfortunately, nobody here is
>> familiar enough with emacs lisp to implement this, but I'd be very glad
>> to help somebody with some emacs lisp skills to work on such a project.
>
> Thanks for the offer. I don't know when I will get to those parts.
> For now, text properties and overlays are not yet supported in the
> modified display engine -- that is the very next problem on my agenda.
> When these parts are done, hopefully others could come on board and
> implement markup support on top of that.
>

--
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp mailto:duerst@...

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-11-27T04:24:32Z

> Date: Fri, 27 Nov 2009 20:11:47 +0900
> From: "Martin J. Dürst" <duerst@...>
> CC: Gregg Reynolds <gar@...>, emacs-bidi@..., emacs-devel@...
>
> On 2009/11/27 1:10, Eli Zaretskii wrote:
> >> Date: Thu, 26 Nov 2009 08:58:39 -0600
> >> From: Gregg Reynolds<gar@...>
>
> >> It's also a major pain to edit XML in a bidi editor. And I do mean
> >> major.
> >
> > Again, an explicit list of main problems would be useful (although I
> > deliberately decided not to handle bidirectional editing for meta
> > documents yet, so this specific class of problems is still a long way
> > from being dealt with).
>
> This is a well-known problem. For some more background and some attempt
> at a solution, please see:
>
> http://www.sw.it.aoyama.ac.jp/2005/pub/IUC28-bidi/ and
> http://www.sw.it.aoyama.ac.jp/2008/pub/IUC32-bidi/

Thanks.

As I pointed out elsewhere, this class of problems is currently beyond
the scope of what I'm trying to do in Emacs. That is because the
display engine basically examines characters one by one, so correct
display of XML/HTML using just UAX#9 algorithm is impossible, as you
and others have found out.

I think that support for bidi editing of documents that use markup
languages should be based on the features that exist as part of the
markup: ‎/‏, the dir= directive etc. Using these, a Lisp
application should place overlays or maybe text properties on the
portions of text that need to be reordered. For example, one could
place an overlay on a portion of the buffer with `display' property
whose value is the reordered text of that portion (it will have to be
enclosed in LRO..PDF embedding to avoid the automatic reordering when
the display string is delivered to the glass). Or we could add a new
overlay property that would just tell the display engine "reorder this
part". Then the display engine will DTRT guided by these overlays and
text properties.

However, please note that the above is not something I thought about
long enough to be convinced that it is the right solution. The only
thing I convinced myself in at this point is that bidi support for
markup languages should not be on the basic display engine level,
which is what I'm currently hacking. Rather, it should use the basic
reordering capabilities of the display engine in some way, but do some
additional work on higher levels to implement the equivalent of the
UAX#9 ``higher protocols''.

> In a discussion with Ken'ichi Handa and Naoto Takahashi in 2005, they
> pointed out that using bidi marks in overlays should make it possible in
> bidi emacs to address this problem. Unfortunately, nobody here is
> familiar enough with emacs lisp to implement this, but I'd be very glad
> to help somebody with some emacs lisp skills to work on such a project.

Thanks for the offer. I don't know when I will get to those parts.
For now, text properties and overlays are not yet supported in the
modified display engine -- that is the very next problem on my agenda.
When these parts are done, hopefully others could come on board and
implement markup support on top of that.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-11-27T03:11:47Z

Hello Eli, Gregg,

On 2009/11/27 1:10, Eli Zaretskii wrote:
>> Date: Thu, 26 Nov 2009 08:58:39 -0600
>> From: Gregg Reynolds<gar@...>

>> It's also a major pain to edit XML in a bidi editor. And I do mean
>> major.
>
> Again, an explicit list of main problems would be useful (although I
> deliberately decided not to handle bidirectional editing for meta
> documents yet, so this specific class of problems is still a long way
> from being dealt with).

This is a well-known problem. For some more background and some attempt
at a solution, please see:

http://www.sw.it.aoyama.ac.jp/2005/pub/IUC28-bidi/ and
http://www.sw.it.aoyama.ac.jp/2008/pub/IUC32-bidi/

Unfortunately, the test prototype available at
http://www.sw.it.aoyama.ac.jp/2008/pub/IUC32-bidi/bidi-source.html
is not very usable currently because the carret jumps around and has to
be repositioned. That problem has been solved, but I didn't get around
to make the update publicly available. Anyway, both the simulation and
the Web-based editor should make it possible to test your problem cases
and tell us whether our solutions help (we know they aren't perfect;
there is no such thing as perfect bidi, unfortunately).

Questions and comments are very welcome!

In a discussion with Ken'ichi Handa and Naoto Takahashi in 2005, they
pointed out that using bidi marks in overlays should make it possible in
bidi emacs to address this problem. Unfortunately, nobody here is
familiar enough with emacs lisp to implement this, but I'd be very glad
to help somebody with some emacs lisp skills to work on such a project.

Regards, Martin.

--
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp mailto:duerst@...

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-11-26T08:10:50Z

> Date: Thu, 26 Nov 2009 08:58:39 -0600
> From: Gregg Reynolds <gar@...>
> Cc: emacs-devel@..., emacs-bidi@...
>
> I've got lots of complaints about the way most bidi-enabled editors handle
> keyboard and cursor interaction; would a list of specific annoyances, er,
> use-cases be of any use?

Yes, certainly. But please try to describe each annoyance, er,
use-case in enough detail, because most probably people here (myself
included) will not be familiar with many other editors. Thanks in
advance.

> For example, in Mellel (http://www.redlers.com/) I've discovered that if an
> RTL document starts with an LTR string, it is impossible to simply insert a
> linefeed in front of that string.

How do you mean ``impossible''? what happens if you try inserting a
newline after going to the beginning of the buffer (with the
equivalent of "M-<" or C-Home)?

> It's also a major pain to edit XML in a bidi editor. And I do mean
> major.

Again, an explicit list of main problems would be useful (although I
deliberately decided not to handle bidirectional editing for meta
documents yet, so this specific class of problems is still a long way
from being dealt with).

Thanks.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-11-26T06:58:39Z

On Sat, Oct 17, 2009 at 3:15 AM, Eli Zaretskii <eliz@...> wrote:

This issue is borderline between design and implementation, but the
decision will affect lots of code to be written/changed later, so I'm
posting this here, in case there are some considerations that I
missed.

I've got lots of complaints about the way most bidi-enabled editors handle keyboard and cursor interaction; would a list of specific annoyances, er, use-cases be of any use?

For example, in Mellel (http://www.redlers.com/) I've discovered that if an RTL document starts with an LTR string, it is impossible to simply insert a linefeed in front of that string. It's also a major pain to edit XML in a bidi editor. And I do mean major. If emacs could make it easy and natural to perform such operations it would be manna from heaven.

Cheers,

Gregg

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Bidi Emacs: R2L paragraphs and the direction of glyph_row

2009-10-17T02:15:18Z

This issue is borderline between design and implementation, but the
decision will affect lots of code to be written/changed later, so I'm
posting this here, in case there are some considerations that I
missed.

Display of paragraphs whose base direction is right-to-left needs to
begin at the right margin of the window, and the empty space at the
end of the line, if any, should extend all the way to the left margin.

By contrast, the terminal-specific back-end (which Emacs uses to
display what's in the glyph matrices) always draws glyphs from left to
right. When the back-end is told that the face should be extended to
the end of the line, it just draws the amount of pixels needed to get
to the margin; the glyph matrix does not tell it how many pixels to
draw. (The text-mode back-end is different, see below.)

To resolve this contradiction, what I plan to do is to introduce a
direction flag into the glyph_row structure. When this flag is
left-to-right, glyphs will start at the lowest address in the
glyph_row->glyphs[] array and will proceed towards the highest
addresses, and the x coordinate will grow in that direction as well,
as it does today. But when this flag is right-to-left, glyphs will
start at the highest address in glyph_row->glyphs[], so that the first
glyph is in glyph_row->glyphs[TEXT_AREA]+glyph_row->used[TEXT_AREA]-1
and the last glyph is in glyph_row->glyphs[TEXT_AREA]. The x
coordinate still grows left to right in this case, as it would on the
screen.

IOW, in the the right-to-left case, the glyphs in the glyph_row are
"reversed". The code which walks glyph rows will need to process such
reversed glyph rows back to front, to keep the rest of the logic
generally intact.

The advantage of this is in the correspondence between the glyph
matrix and what's on the glass. I expect that to minimize changes in
other parts of the code which need to find buffer position that
corresponds to a screen position, or look for glyphs in the matrices
given their screen positions. (The alternative -- to keep glyph rows
in their current left-to-right direction, and reverse them in the
terminal-specific back-end -- sounds less attractive, as it would
require more coding in both the back-ends and in the generic parts of
the display engine.)

This idea of "reversed" glyph rows is already implemented for a
text-mode terminal, and works quite well. Reversing the glyphs in
this case is relatively straightforward, since face extension is
implemented by filling the rest of the line with blanks. After such
filling, reversal is trivial. But for graphics terminals, this is
more complicated, and since I'm not an expert on those back-ends, I
thought I'd ask here if someone sees any reasons why doing the same
would be hard or require many changes in the existing code, even if
that happens only in some very special situations, like displaying
embedded images.

Does someone see any reason why calculating glyph coordinates "in
reverse" or extending the face to the left margin would be hard or
impossible on graphics terminals? Does anyone see problems with the
idea of using "reversed" glyph_row in the display engine?

TIA

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-12T11:39:55Z

> From: Richard Stallman <rms@...>
> CC: emacs-devel@..., emacs-bidi@...
> Date: Mon, 12 Oct 2009 06:11:42 -0400
>
> It is very important to support the full set of features that Emacs
> offers for controlling paragraphs.

OK, I will add this to my TODO.

Thanks.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-12T03:11:42Z

Would it be sufficient to account for any arbitrary amount of
horizontal whitespace between the beginning of the line and the
paragraph regexps?

No, that's not correct. You need to skip whitespace whose width is
the value of `left-margin' and then match the regexp.
(More precisely, you need to skip the amount of space
specified by the value that the function `current-left-margin' would return.)

Looking at the code, I think I was mistaken in what I said about "less
than `left-margin' indentation starts a paragraph". I think that if
the line doesn't have `left-margin' worth of indentation, then the
paragaph regexps match at the end of the indentation.

Look at the code of `forward-paragraph' to see the paragraph criteria
in full detail. It is very important to support the full set of
features that Emacs offers for controlling paragraphs.

At least, it is important to support the full set when this is
released. I won't say it has to be the very next job you work on.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Re: Bidirectional editing in Emacs -- main design decisions

2009-10-11T14:09:34Z

> Date: Sun, 11 Oct 2009 22:12:54 +0200
> From: Eli Zaretskii <eliz@...>
> Cc: emacs-bidi@..., emacs-devel@...
>
> Maybe it will be not hard -- once I make move-to-column,
> current-column, and the rest of indent.c work with bidirectional
> text. Right now, it's badly broken

Broken for a buffer with bidirectional text, I should have said. It
works okay with unidirectional left-to-right text, of course.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-11T13:12:54Z

> From: Richard Stallman <rms@...>
> CC: emacs-devel@..., emacs-bidi@...
> Date: Sun, 11 Oct 2009 04:41:22 -0400
>
> move-to-left-margin shows what it means to be at the left margin.
> It is a matter of matching the paragraph regexps after the right
> amount of whitespace as specified by the value of `left-margin'.

Would it be sufficient to account for any arbitrary amount of
horizontal whitespace between the beginning of the line and the
paragraph regexps? If so, that is an almost trivial modification of
the code I already have.

My problem was with potentially more complicated situations, since
paragraph regexps may in principle be anything.

I also have issues with a paragraph that is separated from the
previous one by just the amount of indentation, like this:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbb

Are you suggesting that Emacs should recompute the paragraph direction
of the two lines with b's, and the result could be a different base
direction from that used by the two preceding lines with a's? I think
that such direction changes will annoy users of bidirectional scripts.
What are the use-cases where such paragraphs are useful?

> When `left-margin' is nonzero, a line which fails to start with that much
> whitespace also starts a paragraph.

You mean, a line which starts with more indentation, or a line that
starts with less? Like this:

aaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaa
xxxxxxxxxxxxxxxxxxxxxxxxxx
yyyyyyyyyyyyyyyyyyyyyy

Should the "xxx" line be considered a new paragraph?

> To be fully correct, it ought to detect paragraphs correctly when
> `left-margin' is nonzero. I think that won't be hard to do.

Maybe it will be not hard -- once I make move-to-column,
current-column, and the rest of indent.c work with bidirectional
text. Right now, it's badly broken, because it assumes buffer
positions increase linearly with screen positions. Even vertical
cursor motion and C-e does not work correctly, because of that.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-11T01:41:22Z

> The only case when the paragraph separator does not begin at the
> beginning of a line is when the left margin is nonzero.

I'm not sure I understand the situation you are describing. (The word
"margin" is too overloaded, even if we confine ourselves to Emacs
parlance alone.) Could you please provide an example of such a
paragraph? Then I could reason about it.

I can't give you an example, but move-to-left-margin shows
what it means to be at the left margin. It is a matter of
matching the paragraph regexps after the right amount of whitespace
as specified by the value of `left-margin'.

When `left-margin' is nonzero, a line which fails to start with that much
whitespace also starts a paragraph.

> Why should these paragraphs be different from other paragraphs
> with regard to direction of text?

They are not "different", they just follow the base direction of the
preceding paragraph.

To be fully correct, it ought to detect paragraphs correctly when
`left-margin' is nonzero. I think that won't be hard to do.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T11:32:10Z

> From: James Cloos <cloos@...>
> Cc: emacs-devel@..., emacs-bidi@...
>
> Thanks for posting that. It is a great summary of the concerns and
> needs of an editor when dealing with bidi test.

Thanks, but I think it's just the beginning. There are lots of other
issues to deal with; see, for example, the aspects of search described
by Ehud Karni in this thread.

The hard problem in making these decisions was to become convinced
that all those other issues are reasonably solvable based on these
basic features, without actually solving any of them.

> Of those points, all but #6 are no brainers; your choices are exactly
> what an editor must do.

Thanks for confirming that.

> Point six is an interesting problem; I'm also unaware of any prior
> art. I suspect that in the long term it would be best to note the
> start and end directionality of such chunks of text and set them
> chunk-by-chunk in a manner similar to how glyphs are set in the
> absence of such properties.

I think this is impossible in general, because once text is reordered,
the information needed to plug in additional chunks (the resolved
level of each character) is lost.

Note that it is fairly simple to reorder the text of `display' strings
together with the surrounding text -- you just need to feed the
characters together into the reordering engine. The problem is
elsewhere -- in the code that uses the produced glyphs.

> But in the short term I agree with the choice you outlined.

The future will tell if it was the right decision. Maybe a useful
first step to examining its validity would be to prepare a fairly
complete list of Emacs applications that currently use the `display'
text properties and overlay properties. Given such a list, one could
think of their applicability to bidirectional editing, and how the
strings should be displayed in each context to do what the users
expect.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T10:18:37Z

>>>>> "Eli" == Eli Zaretskii <eliz@...> writes:

Eli> I'm slowly working on adding support for bidirectional editing in Emacs.

Thanks for posting that. It is a great summary of the concerns and
needs of an editor when dealing with bidi test.

To be fair, I should point out before continuing that I do not read any
rtl scripts. My interests deal with fonts and typography and at least
seeing bidi email in its correct visual order, if only to try to learn
some of it.

Eli> 1. Text storage
Eli> 2. Support for Unicode Bidirectional Algorithm
Eli> 3. Bidi formatting codes are retained
Eli> 4. Reordering of text for display
Eli> 5. Visual-order information is volatile
Eli> 6. Reordering of strings from `display' properties
Eli> 7. Paragraph base direction
Eli> 8. User control of visual order

Of those points, all but #6 are no brainers; your choices are exactly
what an editor must do.

Point six is an interesting problem; I'm also unaware of any prior
art. I suspect that in the long term it would be best to note the
start and end directionality of such chunks of text and set them
chunk-by-chunk in a manner similar to how glyphs are set in the
absence of such properties. But in the short term I agree with
the choice you outlined.

-JimC
--
James Cloos <cloos@...> OpenPGP: 1024D/ED7DAEA6

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T09:38:18Z

> Date: Sat, 10 Oct 2009 16:57:59 +0200
> From: "Ehud Karni" <ehud@...>
> Cc: emacs-bidi@..., emacs-devel@...
>
> On Fri, 09 Oct 2009 23:18:00 Eli Zaretskii wrote:
> >
> > Here's what I can tell about the subject (bidi display) at this point
>
> In general I agree with your decisions.

Well, you brought up many of them (thanks!), so it isn't surprising ;-)

> The search has many problems but this should not influence your bidi
> reordering. The changes to various search functions can be done later.

Agreed.

> The user ALWAYS search for the visual text s/he sees (S/he never knows
> the logical order unless she visits the file literally).

She will look for visual text, but she will type the text she looks
for in the logical (reading) order, not in the visual order, where
characters are reversed and/or reshuffled.

> The problems are caused by many reasons:
> 1. Different logical inputs, even without formatting characters, can
> result in the same visual output.
> e.g. Logical Hebrew text + a number in LTR reading order, the
> number may be before or after the Hebrew text, but in the visual
> output the number will always be after (to the left of) the text.
> Logical "123 HEBREW 456" appears as "123 456 WERBEH".
> 2. Formatting characters are not seen and should not be searched.
> 3. The visual appearance of the searched string may be different from
> what it will match. e.g. The search for logical "HEBREW 3." in
> RTL reading order will appear as ".3 WERBEH" but will match
> also something like logical "HEBREW 3.14159" which its visual
> appearance is "3.14159 WERBEH". This may be what the user wants
> but it may also disturb her because she really wants to find only
> (visual) ".3 WERBEH".

All of these are valid and important considerations, and the search
commands and primitives will have to deal with them, of course.
There's also the issue of ``final'' letters in Hebrew and much more
complex similar issues in Arabic, etc. I hope enough
application-level Emacs programmers will come aboard and handle all
this, because otherwise these scripts will never be supported well
enough in Emacs.

However, taking care of this is still quite far in the future. My
main difficulty in making these decisions was to convince myself that,
while none of these problems are trivial to solve and their solutions
are not even known yet in detail, at least not to me, they are all
_solvable_in_principle_ using just the logical-order text and the
reordering engine (which was designed to allow it to be used by code
other than just the redisplay iterator, so that it's easy to write a
Lisp primitive that takes a logical-order string and returns its
visual-order variant). Comments that question or contradict this
conclusion are what I'm seeking now, because changing these decisions
further down the road may be very difficult, to say nothing of the
wasted effort.

> There is also a technical question, how Emacs will show the found
> string which is not connected as in the "3.14159 WERBEH" above.

I didn't yet adapt support for faces to bidi display. However, my
plan is to make it so that each character produced by the bidi
iterator gets the correct face, like it does today, and faces are (and
will be in the future) set in the logical order of buffer positions.
So, in your example, the characters underlined below will have the
`isearch' face:

3.14159 WERBEH
-- ------

(you were saying that the search string is ".3 WERBEH"). Yes, this
shows as disconnected. But other GUI applications do it that way, so
I think the user will expect this behavior.

> As a minimum adjustment, I think the search must ignore the formatting
> characters.

Yes, of course. At least by default, with an option to not ignore
them.

> Do you intend to support all the explicit formatting characters (LRO is
> specially important as it allows to store visual strings as is) or just
> the implicit (and more used) LRM and RLM ?

All of them. They are already supported in the code that I'm using
now. Like I said, Emacs will support the full set of features
described by UAX#9.

> > This design kills two birds: (a) it produces text that is compliant
> > with other applications, and will display the same as in Emacs, and
> > (b) it avoids the need to invent yet another Emacs infrastructure
> > feature to keep information such as paragraph direction outside of
> > the text itself.
>
> While you can store the LRM and RLM in ISO-8859-8 encoding, there is no
> way to store the the other formatting characters.

UAX#9 recommends to use LRM and RLM, in preference to the other codes,
for this very reason. Users who will want to use the other codes (in
the rare cases where they are necessary), will have to encode text in
UTF-8. I don't see this as a serious problem, though: unlike several
years ago, when this issue was discussed at length on emacs-bidi, the
number of applications supporting UTF-8 is very large today. Heck,
even Notepad groks it nowadays!

> I found an editor that support the all the formatting characters, YODIT
> (http://www.yudit.org/) it is GPLed, may be you can use it.

Thanks, I had it installed already.

> The W3C recommend not to use explicit formatting characters (i.e.
> RLO/LRO/RLE/LRE/PDF) and instead to use markup (see
> http://www.w3.org/International/questions/qa-bidi-controls ,
> specially the "reasons" section).

Yes, I know. The obsession of W3C with markup is well known ;-) But
Emacs is first and foremost a _text_editor_, so it doesn't make sense
to me to force users to use markup just to be able to read or write
bidirectional text. I also believe that converting text that uses
Unicode formatting codes into markup is not such a hard job, and
someone will surely come up soon enough with an Emacs function to do
that.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T09:06:47Z

> Date: Sat, 10 Oct 2009 23:13:48 +0800
> From: Jason Rumney <jasonr@...>
> CC: emacs-devel@..., emacs-bidi@...
>
> Eli Zaretskii wrote:
> > 4. Reordering of text for display
> >
>
> Does the function font-shape-gstring help with fitting this in?

I'm sorry to say that I don't understand what it does. If you can
explain, or give me an educational example to play with, maybe I will
be able to answer your question, my profound ignorance of GUI
rendering notwithstanding.

> > 8. User control of visual order
> >
> > I decided it was unjustified to deviate from UAX#9. Its algorithm
> > already provides the solution to this problem: users can always
> > control the visual order by inserting special formatting codes at
> > strategic places.
>
> Couldn't Emacs by default use the clever heuristics to decide when to
> automatically insert the special formatting codes? It would have to be
> optional and undoable of course, because heuristics are never perfect,
> but it seems to me as a naive non-speaker of RTL languages that to DWIM
> in these edge cases is the right behaviour.

I agree: if it's possible to DWIM automatically, we should. But you
are talking about higher levels than where I am right now. For my
purposes, it was good enough to decide that any such clever heuristics
will eventually just add or remove formatting codes, and that no other
infrastructure features are needed to support this.

> Also you mention several times that the special direction change codes
> are not displayed, but there should be an option to display them IMHO,
> (perhaps part of whitespace.el) as users may need to distinguish between
> explicit direction changes and implicit ones in some circumstances.

Yes, that's a very important feature, of course. Which is why I
tried (but obviously failed, since you are the second person asking
the same) to tell that it will and must be supported:

In addition, being able to show these formatting codes to the user
is a valuable feature, because the way reordered text looks might
not be otherwise understood or changed easily.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T07:57:59Z

On Fri, 09 Oct 2009 23:18:00 Eli Zaretskii wrote:
>
> Here's what I can tell about the subject (bidi display) at this point

In general I agree with your decisions.

> 1. Text storage
>
> Bidirectional text in Emacs buffers and strings is stored in strict
> logical order (a.k.a. "reading order"). This is how most (if not
> all) other implementations handle bidirectional text. The
> advantage of this is that file and process I/O is trivial, as well
> as text search. [snip]

The search has many problems but this should not influence your bidi
reordering. The changes to various search functions can be done later.

The user ALWAYS search for the visual text s/he sees (S/he never knows
the logical order unless she visits the file literally).

The problems are caused by many reasons:
1. Different logical inputs, even without formatting characters, can
result in the same visual output.
e.g. Logical Hebrew text + a number in LTR reading order, the
number may be before or after the Hebrew text, but in the visual
output the number will always be after (to the left of) the text.
Logical "123 HEBREW 456" appears as "123 456 WERBEH".
2. Formatting characters are not seen and should not be searched.
3. The visual appearance of the searched string may be different from
what it will match. e.g. The search for logical "HEBREW 3." in
RTL reading order will appear as ".3 WERBEH" but will match
also something like logical "HEBREW 3.14159" which its visual
appearance is "3.14159 WERBEH". This may be what the user wants
but it may also disturb her because she really wants to find only
(visual) ".3 WERBEH".
There is also a technical question, how Emacs will show the found
string which is not connected as in the "3.14159 WERBEH" above.

As a minimum adjustment, I think the search must ignore the formatting
characters. An option to show (or operate, in search & replace) only on
found matches that are also the same visually is recommended.

> 3. Bidi formatting codes are retained

Agreed, but see my comment on search.

> 7. Paragraph base direction
>
> There is a buffer-specific variable `paragraph-direction' that
> allows to override this dynamic detection of the direction of each
> paragraph, and force a certain base direction on all paragraphs in
> the buffer. I expect, for example, each major mode for a
> programming language to force the left-to-right paragraph
> direction, because programming languages are written left to right,
> and right-to-left scripts appear in such buffers only in strings
> embedded in the program or in comments.

I think a better name is `bidi-paragraphs-direction' or even
`bidi-paragraphs-reading-direction'. Note the `s' in paragraphs,
because it is influence all the paragraphs in the buffer.

There should be a key to toggle this variable. It will very
useful for the minibuffer.

> 8. User control of visual order

Do you intend to support all the explicit formatting characters (LRO is
specially important as it allows to store visual strings as is) or just
the implicit (and more used) LRM and RLM ?

> This design kills two birds: (a) it produces text that is compliant
> with other applications, and will display the same as in Emacs, and
> (b) it avoids the need to invent yet another Emacs infrastructure
> feature to keep information such as paragraph direction outside of
> the text itself.

While you can store the LRM and RLM in ISO-8859-8 encoding, there is no
way to store the the other formatting characters.

> That is all for now. If you have comments or questions, you are
> welcome to voice them.

I found an editor that support the all the formatting characters, YODIT
(http://www.yudit.org/) it is GPLed, may be you can use it.

The W3C recommend not to use explicit formatting characters (i.e.
RLO/LRO/RLE/LRE/PDF) and instead to use markup (see
http://www.w3.org/International/questions/qa-bidi-controls ,
specially the "reasons" section).

Ehud.

--
Ehud Karni Tel: +972-3-7966-561 /"\
Mivtach - Simon Fax: +972-3-7976-561 \ / ASCII Ribbon Campaign
Insurance agencies (USA) voice mail and X Against HTML Mail
http://www.mvs.co.il FAX: 1-815-5509341 / \
GnuPG: 98EA398D <http://www.keyserver.net/> Better Safe Than Sorry

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T07:05:34Z

> From: Sascha Wilde <wilde@...>
> Cc: emacs-devel@..., emacs-bidi@...
> Date: Sat, 10 Oct 2009 15:44:19 +0200
>
> Eli Zaretskii <eliz@...> wrote:
> > 8. User control of visual order
> [...]
> > Emacs could
> > have a command called, say, `make-paragraph-left-to-right' that did
> > its job simply by inserting LRM at the beginning of the paragraph.
>
> I would suggest that Emacs should also have a way to visualize the
> otherwise invisible text direction marks so that:
>
> - it becomes transparent to the user whether the direction of an
> specific portion of the text is explicit or implicit defined
>
> - the user is provided with a simple way to remove the marks in case he
> wants to. (By simply deleting them the way he would do with any other
> regular character.)

Yes. That's what this part of my longish message was trying to say:

In addition, being able to show these formatting codes to the user
is a valuable feature, because the way reordered text looks might
not be otherwise understood or changed easily.

The "direction marks" you mention are the "formatting codes" I wrote
about.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T04:36:38Z

> From: Richard Stallman <rms@...>
> CC: emacs-devel@..., emacs-bidi@...
> Date: Sat, 10 Oct 2009 05:16:58 -0400
>
> The reason for this deliberate deviation from the letter of Emacs
> definition of a paragraph are complicated, but the upshot is that from
> the user point of view, it does not make sense to change paragraph
> direction if the paragraph separator does not begin at the beginning
> of a line.
>
> The only case when the paragraph separator does not begin at the
> beginning of a line is when the left margin is nonzero.

I'm not sure I understand the situation you are describing. (The word
"margin" is too overloaded, even if we confine ourselves to Emacs
parlance alone.) Could you please provide an example of such a
paragraph? Then I could reason about it.

> Why should these paragraphs be different from other paragraphs
> with regard to direction of text?

They are not "different", they just follow the base direction of the
preceding paragraph.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T02:16:58Z

The reason for this deliberate deviation from the letter of Emacs
definition of a paragraph are complicated, but the upshot is that from
the user point of view, it does not make sense to change paragraph
direction if the paragraph separator does not begin at the beginning
of a line.

The only case when the paragraph separator does not begin at the
beginning of a line is when the left margin is nonzero.

Why should these paragraphs be different from other paragraphs
with regard to direction of text?

As another deviation from the definition of a paragraph, text that
matches `paragraph-separate' is given the same direction as the
preceding paragraph. (By contrast, Emacs generally does not consider
`paragraph-separate' as part of any paragraph.)

I don't think that conflicts at all with the normal definition
of paragraphs. The separator isn't part of the paragraph,
but its reading direction needs to be determined somehow.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T01:18:59Z

> From: joakim@...
> Cc: emacs-devel@..., emacs-bidi@...
> Date: Sat, 10 Oct 2009 09:28:19 +0200
>
> Eli Zaretskii <eliz@...> writes:
>
> >> From: joakim@...
> >> Cc: emacs-devel@..., emacs-bidi@...
> >> Date: Sat, 10 Oct 2009 00:42:39 +0200
> >>
> >> Presumably I will also need to tell the widget to render its own text in
> >> bidi mode.
> >
> > By itself, or by using Emacs facilities?
>
> They are gtk widgets. I havent looked at it closely, but I presume you
> tell the gtk widgets which locale to render text in.

Actually, I'd expect gtk widgets to do this automatically, no matter
in what locale. The text it renders should supply the hint.

> So, by itself, yes.

You can do that, Emacs won't care.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-10T00:08:54Z

> From: joakim@...
> Cc: emacs-devel@..., emacs-bidi@...
> Date: Sat, 10 Oct 2009 00:42:39 +0200
>
> Presumably I will also need to tell the widget to render its own text in
> bidi mode.

By itself, or by using Emacs facilities?

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-09T15:41:51Z

> Date: Fri, 09 Oct 2009 23:18:00 +0200
> From: Eli Zaretskii <eliz@...>
> Cc:
>
> So I decided to use such a higher protocol -- namely,
> the Emacs definition of a paragraph, as determined by the
> `paragraph-start' and `paragraph-separate' regexps.

A small, but significant correction to this: these two regexps are
looked for anchored at line beginning.

The reason for this deliberate deviation from the letter of Emacs
definition of a paragraph are complicated, but the upshot is that from
the user point of view, it does not make sense to change paragraph
direction if the paragraph separator does not begin at the beginning
of a line.

As another deviation from the definition of a paragraph, text that
matches `paragraph-separate' is given the same direction as the
preceding paragraph. (By contrast, Emacs generally does not consider
`paragraph-separate' as part of any paragraph.)

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Bidirectional editing in Emacs -- main design decisions

2009-10-09T15:29:24Z

> From: joakim@...
> Cc: emacs-devel@..., emacs-bidi@...
> Date: Fri, 09 Oct 2009 23:55:19 +0200
>
> It works mostly the same as embedding images. From what youre
> writing below it sounds like the display of images will work as
> before, therefore my patch will apply, hopefully nicely, on top of
> bidi. Correct?

Correct. Images and any other objects will be reordered according to
UAX#9, and as a single entity. IOW, imagine that instead of the
embedded widget the buffer has a single character U+FFFC (OBJECT
REPLACEMENT CHARACTER). The reordering code will treat the embedded
widget as it would treat that character. That means, in particular,
that if the widget is embedded in text written in some right-to-left
script, the text that precedes the widget will be on the right of the
widget, and text that follows it will be on the left.

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Bidirectional editing in Emacs -- main design decisions

2009-10-09T14:18:00Z

As some of you know, I'm slowly working on adding support for
bidirectional editing in Emacs. (Before you ask: the code is not
publicly available yet, and won't be until Emacs switches to bzr as
its main VCS.)

While there's a lot of turf to be covered yet, I thought I'd publish
the main design decisions up to this point. Many of these decisions
were discussed at length years ago on emacs-bidi mailing list, and
since then I also talked them over in private email with a few people.
Other decisions were made recently, as I went about changing the
display engine.

My goal, and the main drive behind these design decisions was to
preserve as much as possible the basic assumptions and design
principles of the current Emacs display engine. This is not just
opportunism; I firmly believe that any other way would mean a total
redesign and rewrite of the display engine, which is something we want
to avoid. Personally, if such a redesign would be necessary, I
couldn't have participated in that endeavor, except as advisor.

With that preamble out of my way, here's what I can tell about the
subject at this point:

1. Text storage

Bidirectional text in Emacs buffers and strings is stored in strict
logical order (a.k.a. "reading order"). This is how most (if not
all) other implementations handle bidirectional text. The
advantage of this is that file and process I/O is trivial, as well
as text search. The disadvantage is that text needs to be
reordered for display (see below) and also for sending to any other
visual-order stream, such as a printer or a file in visual-order
encoding.

2. Support for Unicode Bidirectional Algorithm

The Unicode Bidirectional Algorithm, described in Annex 9 of the
Unicode Standard (a.k.a. UAX#9, see http://www.unicode.org/reports/tr9/),
specifies how to reorder bidirectional text from logical to visual
order. Emacs will belong to the so-called "Full Bidirectionality"
class of applications, which include support for both implicit
bidirectional reordering and explicit directional embedding codes
that allow to override the implicit reordering. This means that
Emacs supports the entire spectrum of Unicode character properties
and special codes relevant to bidirectional text.

3. Bidi formatting codes are retained

At some point in the reordering described by UAX#9, the various
formatting codes are to be removed from the text, once they've
performed their role of forcing the order of characters for
display, because they are not supposed to be visible on display.
Contrary to this, Emacs does not remove these formatting codes, it
just behaves as if they are not there. (This behavior is
acknowledged by UAX#9 under "Retaining Format Codes" clause, so
Emacs does not break conformance here.) This is primarily because
Emacs must preserve the text that was not edited; in particular,
visiting a file and then saving it to a different file without
changing anything must produce the same byte stream as the original
file, even if the formatting codes were part of the original file.
In addition, being able to show these formatting codes to the user
is a valuable feature, because the way reordered text looks might
not be otherwise understood or changed easily.

4. Reordering of text for display

Reordering for display happens as part of Emacs redisplay. In a
nutshell, the current unidirectional redisplay code walks through
buffer text and considers each character in turn. After each
character is processed and translated into a `struct glyph', which
includes all the information needed for displaying that character,
the iterator's position is incremented to the next character.

In the bidi Emacs, this _linear_ iteration through the buffer is
replaced with a _non-linear_ one, whereby instead of incrementing
buffer position, a function is called to return the next position
in the visual order. Whatever position it returns is processed
next into a `struct glyph'. The rest of the code that produces
"glyph matrices" (data structures used to decide which parts of the
screen need to be redrawn) is largely ignorant of the
bidirectionality of the text. Of course, parts of the display
engine that manipulate the glyph matrices directly and assume that
buffer positions increase monotonically with glyph positions need
to be fixed or rewritten. But these parts of the display are
relatively few and localized. Also, some redisplay optimizations
need to be disabled when bidirectional text is rendered for
display.

5. Visual-order information is volatile

There were lots of discussions several years ago about whether
Emacs should record in some way the information needed to reorder
text into visual order of the characters, to reuse it later. In
UAX#9 terminology, this information is the "resolved level" of each
character. Various features were suggested as a vehicle for this,
for example, some special text properties (except that text
properties, unlike resolved levels, cannot overlap). Lots of
energy went into discussing how this information would be recorded
and how it will be reused, e.g. if portion of the text was
copy-pasted into a different buffer or string. The complications,
it turns out, are abound.

The current design doesn't record this information at all. It
simply recomputes it each time a buffer or string need to be
displayed or sent to a visual-order stream. The resolved levels
are computed during reordering, then forgotten. It turns out that
bidirectional iteration through buffer text is not much more
expensive than the current unidirectional one. The implementation
of UAX#9 written for Emacs is efficient enough to make any
long-term caching of resolved levels unnecessary.

6. Reordering of strings from `display' properties

Strings that are values of `display' text properties and overlay
properties are reordered individually. This matters when such
properties cover adjacent portions of buffer text, back to back.
For example, PROP1 is associated with buffer positions P1 to P2,
and PROP2 immediately follows it, being associated with positions
P2 to P3. The current design calls for reordering the characters
of the strings that are the values of PROP1 and PROP2 separately.
An alternative would be to feed them concatenated into the
reordering algorithm, in which case the characters coming from
PROP2 could end up displayed before (to the left) of the characters
coming from PROP1. However, this alternative requires a major
surgery of several parts of the display code. (Interested readers
are advised to read the code of set_cursor_from_row in xdisp.c, as
just one example.) It's not clear what is TRT to do in this case
anyway; I'm not aware of any other application that provides
similar features, so there's nothing I could compare it to. So I
decided to go with the easier design. If the application needs a
single long string, it can always collapse two or more `display'
properties into one long one.

Another, perhaps more serious implication of this design decision
is that strings from `display' properties are reordered separately
from the surrounding buffer text. IOW, production of glyphs from
reordered buffer text is stopped when a `display' property is
found, the string that is the property's value is reordered and
displayed, and then the rest of text is reordered and its glyphs
produced. The effect will be visible, e.g., when a `display'
string is embedded in right-to-left text in otherwise left-to-right
paragraph text. Again, I think in the absence of clear "prior
art", simplicity of design and the amount of changes required in
the existing display engine win here.

7. Paragraph base direction

Bidirectional text can be rendered in left-to-right or in
right-to-left paragraphs. The former is used for mostly
left-to-right text, possibly with some embedded right-to-left text.
The latter is used for text that is mostly or entirely
right-to-left. Right-to-left paragraphs are displayed flushed all
the way to the right margin of the display; this is how users of
right-to-left scripts expect to see text in their languages.

UAX#9 specifies how to determine whether this attribute of a
paragraph, called "base direction", is one or the other, by finding
the first strong directional character in the paragraph. However,
the Unicode Character Database specifies that NL and CR characters
are paragraph separators, which means each line is a separate
paragraph, as far as UAX#9 is concerned. If Emacs would follow
UAX#9 to the letter, each line could have different base direction,
which is, of course, intolerable. We could avoid this nonsense by
using the "soft newline" or similar features, but I firmly believe
that Emacs should DTRT with bidirectional text even in the simplest
modes, including the Fundamental mode, where every newline is hard.

Fortunately, UAX#9 acknowledges that applications could have other
ideas about what is a "paragraph". It calls this ``higher
protocol''. So I decided to use such a higher protocol -- namely,
the Emacs definition of a paragraph, as determined by the
`paragraph-start' and `paragraph-separate' regexps. Therefore, the
first strong directional character after `paragraph-start' or
`paragraph-separate' determines the paragraph direction, and that
direction is kept for all the lines of the paragraph, until another
`paragraph-separate' is found. (Of course, this means that
inserting a single character near the beginning of a paragraph
might affect the display of all the lines in that paragraph, so
some of the current redisplay optimizations which deal with changes
to a single line need to be disabled in this case.)

There is a buffer-specific variable `paragraph-direction' that
allows to override this dynamic detection of the direction of each
paragraph, and force a certain base direction on all paragraphs in
the buffer. I expect, for example, each major mode for a
programming language to force the left-to-right paragraph
direction, because programming languages are written left to right,
and right-to-left scripts appear in such buffers only in strings
embedded in the program or in comments.

8. User control of visual order

UAX#9 does not always produce perfect results on the screen.
Notable cases where it doesn't are related to characters such as
`+' and `-' which have more than one role: they can be used in
mathematical context or in plain-text context; the "correct"
reordering turns out to be different in each case.

Again, lots of energy was invested in past discussions how to
prevent these blunders. Several clever heuristics are known to
avoid that. The problem is that all those heuristics contradict
UAX#9, which means text that looks OK in Emacs will look different
(i.e. wrong) in another application.

I decided it was unjustified to deviate from UAX#9. Its algorithm
already provides the solution to this problem: users can always
control the visual order by inserting special formatting codes at
strategic places. These codes are by default not shown in the
displayed text, but they influence the resolved directionality of
the surrounding characters, and thus change their visual order. We
could (and probably should) have commands in Emacs to control the
visual order that will work simply by inserting the appropriate
formatting codes. For example, a paragraph starting with an Arabic
letter could nonetheless be rendered as left-to-right paragraph by
inserting the LRM code before that Arabic character; Emacs could
have a command called, say, `make-paragraph-left-to-right' that did
its job simply by inserting LRM at the beginning of the paragraph.

This design kills two birds: (a) it produces text that is compliant
with other applications, and will display the same as in Emacs, and
(b) it avoids the need to invent yet another Emacs infrastructure
feature to keep information such as paragraph direction outside of
the text itself.

That is all for now. If you have comments or questions, you are
welcome to voice them. However, I reserver the right to respond only
to those I'm interested in and/or have time for. ;-)

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Compilation error 0.9.1

2009-07-20T21:59:03Z

Hajder writes:

> Regarding this thread:
> http://www.nabble.com/Query-regarding-emacs-compilation-errors.-td17102487.html

> For your information, just wanted to say that the (second) compilation
> error, is also apparent on my linux desktop - Ubuntu Jaunty Jackalope 9.04

Try the following commands after the segmentation fault caused by
"make".

$ cd src
$ setarch i386 -R ./temacs --batch --load loadup dump

I found this workaround in the etc/PROBLEMS file. I have just tried
this method on Fedora 11 and Ubuntu 9.04. It worked.

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Compilation error 0.9.1

2009-07-19T05:38:13Z

Hi

Regarding this thread: http://www.nabble.com/Query-regarding-emacs-compilation-errors.-td17102487.html

For your information, just wanted to say that the (second) compilation error, is also apparent on my linux desktop - Ubuntu Jaunty Jackalope 9.04

Loading float-sup...
((43812 . 5614) (8840 . 0) (539 . 82) 60720 139632 (14 . 9) (14 . 12) (4426 . 1685))
Loading vc-hooks...
Loading ediff-hook...
((44774 . 4652) (8940 . 0) (540 . 81) 61640 139649 (14 . 9) (14 . 12) (4486 . 1625))
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under names emacs and emacs-21.3.50.1
Segmentation fault
make[1]: *** [emacs] Error 139
make[1]: Leaving directory `/home/hajder/emacs-bidi/src'
make: *** [src] Error 2

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Query regarding emacs compilation errors.

2009-07-16T05:20:04Z

Michael Blaustein writes:

> Was this problem with compiling emacs-bidi-0.9.1 on fedora ever solved?

> I have the same problem on fedora 10 and fedora 11. I'll appreciate any
> advice.

Not yet. We are trying to upgrade it.

> On Thu, 2008-05-08 at 16:40 +0900, TAKAHASHI Naoto wrote:
>> "Manjiri Gapchup" writes:
>>
>> > My linux version is
>>
>> > Linux torana.trinity.local 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12
>> > 19:40:16 EDT 2007 i686 i686 i386 GNU/Linux
>>
>> It seems that you are using Fedora 5. I have just tried to compile
>> emacs-bidi-0.9.1 on Fedora 7 and Fedra 8, and they both returned the
>> same error that you reported. :-(
>>
>> > Will you please let us know which Linux version you are using. We will
>> > try to install it here and test.
>>
>> I am using Debian Sid (Unstable), but I have confirmed that
>> emacs-bidi-0.9.1 compiles fine on Debian Etch (Stable), too. I
>> recommend you to try Etch first.
>>
>> We will investigate the cause of the error with Fedora systems as soon
>> as the main author of the emacs-bidi comes back from his business
>> trip.
>>

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Query regarding emacs compilation errors.

2009-07-15T08:36:57Z

Was this problem with compiling emacs-bidi-0.9.1 on fedora ever solved?

I have the same problem on fedora 10 and fedora 11. I'll appreciate any
advice.

Thanks.

On Thu, 2008-05-08 at 16:40 +0900, TAKAHASHI Naoto wrote:

> "Manjiri Gapchup" writes:
>
> > My linux version is
>
> > Linux torana.trinity.local 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12
> > 19:40:16 EDT 2007 i686 i686 i386 GNU/Linux
>
> It seems that you are using Fedora 5. I have just tried to compile
> emacs-bidi-0.9.1 on Fedora 7 and Fedra 8, and they both returned the
> same error that you reported. :-(
>
> > Will you please let us know which Linux version you are using. We will
> > try to install it here and test.
>
> I am using Debian Sid (Unstable), but I have confirmed that
> emacs-bidi-0.9.1 compiles fine on Debian Etch (Stable), too. I
> recommend you to try Etch first.
>
> We will investigate the cause of the error with Fedora systems as soon
> as the main author of the emacs-bidi comes back from his business
> trip.
>

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

"make" failure for emacs-bidi-0.9.1.tar.gz

2008-06-02T06:47:41Z

After unpacking the above file, I followed these instructions:

$ cd emacs-bidi
$ ./configure
$ make

This is what happened:

sudo make
Password:
cd lib-src; make all CC='gcc' CFLAGS='-g -O2'
CPPFLAGS='-I/usr/X11R6/include -I/usr/local/include -L/usr/local/lib'
LDFLAGS='-L/usr/local/lib' MAKE='make'
cd src; make all CC='gcc' CFLAGS='-g -O2'
CPPFLAGS='-I/usr/X11R6/include -I/usr/local/include -L/usr/local/lib'
LDFLAGS='-L/usr/local/lib' MAKE='make'
cd ../lwlib/; make CC='gcc' CFLAGS='-g -O2' MAKE='make'
"C_SWITCH_X_SITE=-I/usr/local/include" "C_SWITCH_X_MACHINE="
"C_SWITCH_X_SYSTEM=" "C_SWITCH_SITE=" "C_SWITCH_MACHINE="
"C_SWITCH_SYSTEM=-I/usr/X11R6/include -I/usr/local/include
-L/usr/local/lib"
touch stamp-oldxmenu
gcc -c -I/usr/X11R6/include -I/usr/local/include -L/usr/local/lib
-Demacs -DHAVE_CONFIG_H -DUSE_LUCID -I.
-I/home/john/Desktop/emacs-bidi/src -I/usr/X11R6/include
-I/usr/local/include -L/usr/local/lib -I/usr/local/include -g -O2
xterm.c
xterm.c: In function `x_set_toolkit_scroll_bar_thumb':
xterm.c:9181: error: structure has no member named `scroll_mode'
xterm.c:9183: error: structure has no member named `scroll_mode'
xterm.c:9194: error: structure has no member named `scroll_mode'
*** Error code 1

Stop in /usr/home/john/Desktop/emacs-bidi/src.
*** Error code 1

Stop in /usr/home/john/Desktop/emacs-bidi.

Can anybody help? I am running PC-BSD and I suspect I need to do
something about how it implements x windows, but don't know what or
how.

thanks

John

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Query regarding emacs compilation errors.

2008-05-08T00:40:08Z

"Manjiri Gapchup" writes:

> My linux version is

> Linux torana.trinity.local 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12
> 19:40:16 EDT 2007 i686 i686 i386 GNU/Linux

It seems that you are using Fedora 5. I have just tried to compile
emacs-bidi-0.9.1 on Fedora 7 and Fedra 8, and they both returned the
same error that you reported. :-(

> Will you please let us know which Linux version you are using. We will
> try to install it here and test.

I am using Debian Sid (Unstable), but I have confirmed that
emacs-bidi-0.9.1 compiles fine on Debian Etch (Stable), too. I
recommend you to try Etch first.

We will investigate the cause of the error with Fedora systems as soon
as the main author of the emacs-bidi comes back from his business
trip.

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

RE: Query regarding emacs compilation errors.

2008-05-07T23:31:31Z

Will you please let us know which Linux version you are using. We will
try to install it here and test.

Thanks and Regards,
Manjiri

-----Original Message-----
From: Manjiri Gapchup
Sent: Thursday, May 08, 2008 9:14 AM
To: 'TAKAHASHI Naoto'
Cc: emacs-bidi@...; Kalpesh Balar
Subject: RE: [emacs-bidi] Query regarding emacs compilation errors.

Hi,

My linux version is

Linux torana.trinity.local 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12
19:40:16 EDT 2007 i686 i686 i386 GNU/Linux

Please let me know if any more information is required.

Thanks and Regards,
Manjiri

-----Original Message-----
From: TAKAHASHI Naoto [mailto:ntakahas@...]
Sent: Thursday, May 08, 2008 9:07 AM
To: Manjiri Gapchup
Cc: emacs-bidi@...; Kalpesh Balar
Subject: Re: [emacs-bidi] Query regarding emacs compilation errors.

"Manjiri Gapchup" writes:

> And the previous error is not seen but now I am getting Error as,

> Loading vc-hooks...
> Loading ediff-hook...
> ((44753 . 4669) (8940 . 0) (540 . 81) 61678 139650 (14 . 9) (14 . 12)
> (4483 . 1565)) Finding pointers to doc strings...
> Finding pointers to doc strings...done Dumping under names emacs and
> emacs-21.3.50.1
> make[1]: *** [emacs] Segmentation fault
> make[1]: *** Deleting file `emacs'
> make[1]: Leaving directory `/home/mgapchup/emacs-bidi/src'
> make: *** [src] Error 2

I could not reproduce this error. Will you tell me about your "linux"
system in more detail? For example the distribution name, the version
number, CPU architecture, etc.

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

RE: Query regarding emacs compilation errors.

2008-05-07T20:44:03Z

Hi,

My linux version is

Linux torana.trinity.local 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12
19:40:16 EDT 2007 i686 i686 i386 GNU/Linux

Please let me know if any more information is required.

Thanks and Regards,
Manjiri

-----Original Message-----
From: TAKAHASHI Naoto [mailto:ntakahas@...]
Sent: Thursday, May 08, 2008 9:07 AM
To: Manjiri Gapchup
Cc: emacs-bidi@...; Kalpesh Balar
Subject: Re: [emacs-bidi] Query regarding emacs compilation errors.

"Manjiri Gapchup" writes:

> And the previous error is not seen but now I am getting Error as,

Re: Query regarding emacs compilation errors.

2008-05-07T20:36:54Z

"Manjiri Gapchup" writes:

> And the previous error is not seen but now I am getting
> Error as,

> Loading vc-hooks...
> Loading ediff-hook...
> ((44753 . 4669) (8940 . 0) (540 . 81) 61678 139650 (14 . 9) (14 . 12) (4483 . 1565))
> Finding pointers to doc strings...
> Finding pointers to doc strings...done
> Dumping under names emacs and emacs-21.3.50.1
> make[1]: *** [emacs] Segmentation fault
> make[1]: *** Deleting file `emacs'
> make[1]: Leaving directory `/home/mgapchup/emacs-bidi/src'
> make: *** [src] Error 2

RE: Query regarding emacs compilation errors.

2008-05-07T19:46:23Z

Hi,

Thanks for your support.

I have modified
Line 3371 in keyboard.c from

static SIGTYPE interrupt_signal P_ ((int));
to
extern SIGTYPE interrupt_signal ();
And the previous error is not seen but now I am getting
Error as,

Loading vc-hooks...
Loading ediff-hook...
((44753 . 4669) (8940 . 0) (540 . 81) 61678 139650 (14 . 9) (14 . 12) (4483 . 1565))
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under names emacs and emacs-21.3.50.1
make[1]: *** [emacs] Segmentation fault
make[1]: *** Deleting file `emacs'
make[1]: Leaving directory `/home/mgapchup/emacs-bidi/src'
make: *** [src] Error 2

Will you please guide me in this context ?

Thanks and Regards,
Manjiri

-----Original Message-----
From: TAKAHASHI Naoto [mailto:ntakahas@...]
Sent: Wednesday, May 07, 2008 6:25 PM
To: Manjiri Gapchup
Cc: emacs-bidi@...; Kalpesh Balar
Subject: Re: [emacs-bidi] Query regarding emacs compilation errors.

"Manjiri Gapchup" writes:

> We have extracted emacs-bidi-0.9.1.tar.gz <http://www.m17n.org/emacs-bidi/emacs-bidi-0.9.1.tar.gz> and executed on linux

> 1) ./configure
> 2) make

> While compilation process we are getting following errors

> keyboard.c: In function âkbd_buffer_store_eventâ:
> keyboard.c:3371: error: invalid storage class for function
> âinterrupt_signalâ
> keyboard.c: At top level:
> keyboard.c:9925: warning: conflicting types for âinterrupt_signalâ
> keyboard.c:3415: warning: previous implicit declaration of
> âinterrupt_signalâ was here
> make[1]: *** [keyboard.o] Error 1
> make[1]: Leaving directory `/home/mgapchup/emacs-bidi/src'
> make: *** [src] Error 2

> Will you please guide us so that we can run this code?

Please change Line 3371 in keyboard.c from

static SIGTYPE interrupt_signal P_ ((int));

to
extern SIGTYPE interrupt_signal ();

Sorry for inconvenience.

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi

Re: Query regarding emacs compilation errors.

2008-05-07T05:54:31Z

"Manjiri Gapchup" writes:

> We have extracted emacs-bidi-0.9.1.tar.gz <http://www.m17n.org/emacs-bidi/emacs-bidi-0.9.1.tar.gz> and executed on linux

> 1) ./configure
> 2) make

> While compilation process we are getting following errors

> keyboard.c: In function âkbd_buffer_store_eventâ:
> keyboard.c:3371: error: invalid storage class for function âinterrupt_signalâ
> keyboard.c: At top level:
> keyboard.c:9925: warning: conflicting types for âinterrupt_signalâ
> keyboard.c:3415: warning: previous implicit declaration of âinterrupt_signalâ was here
> make[1]: *** [keyboard.o] Error 1
> make[1]: Leaving directory `/home/mgapchup/emacs-bidi/src'
> make: *** [src] Error 2

> Will you please guide us so that we can run this code?

Please change Line 3371 in keyboard.c from

static SIGTYPE interrupt_signal P_ ((int));

to
extern SIGTYPE interrupt_signal ();

Sorry for inconvenience.

--
TAKAHASHI Naoto
ntakahas@...
http://www.m17n.org/ntakahas/

_______________________________________________
emacs-bidi mailing list
emacs-bidi@...
http://lists.gnu.org/mailman/listinfo/emacs-bidi