Nabble - Free Desktop - fribidi

Re: conjunction between chars

2009-11-20T14:35:09Z

Hi Milad,

Are you using FriBidi 0.19.x? The 0.10 series don't do Arabic shaping.

behdad

On 11/20/2009 05:10 PM, Milad Rastian wrote:
> Hi fribidi !
> I have a little code which use fribid lib(attached that), it works. but
> when I test it in gnome-terminal chars are not conjunction together,
> should I use another function to conjunct chars ?
>
>
>
>
> Thanks,
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

conjunction between chars

2009-11-20T14:10:28Z

Hi fribidi !
I have a little code which use fribid lib(attached that), it works. but when I test it in gnome-terminal chars are not conjunction together,
should I use another function to conjunct chars ?

Thanks,
--
/--------------------
Milad Rastian
http://milad.rastian.com

[milad02.c]

#include "fribidi.h" #include <string.h> #include <stdio.h> #include <stdlib.h> //`#include <stdarg.h> #define MAX_STR_LEN 65000 //gcc tests.c -lfribidi -o milad #define ALLOCATE(tp,ln) ((tp *) malloc (sizeof (tp) * (ln))) void main(){ FriBidiChar *u_logical_str, *u_visual_str; /* unicode strings .... */ int len, alloc_len; FriBidiCharType base_dir; FriBidiStrIndex *position_L_to_V_list; FriBidiStrIndex *position_V_to_L_list; FriBidiLevel *embedding_level_list; base_dir = FRIBIDI_TYPE_RTL; char *out_string; char *in_string="من می رم"; len = strlen(in_string); alloc_len = len+1; u_logical_str = ALLOCATE(FriBidiChar,alloc_len); u_logical_str = ALLOCATE (FriBidiChar,alloc_len); u_visual_str = ALLOCATE (FriBidiChar,alloc_len); position_L_to_V_list = ALLOCATE (FriBidiStrIndex,alloc_len); position_V_to_L_list = ALLOCATE (FriBidiStrIndex,alloc_len); embedding_level_list = ALLOCATE(FriBidiLevel,alloc_len); if(in_string[len-1] == '\n') { in_string[len-1] = '\0'; } int char_set_num; char *char_set; char_set = "UTF-8"; char_set_num = fribidi_parse_charset (char_set); len = fribidi_charset_to_unicode(char_set_num, in_string, len, u_logical_str); out_string = ALLOCATE(char,alloc_len); fribidi_log2vis(u_logical_str, len, &base_dir, u_visual_str, position_L_to_V_list, position_V_to_L_list, embedding_level_list); fribidi_unicode_to_charset(char_set_num, u_visual_str, len, out_string); printf("%s\n",out_string); }

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Implementation of X6 in the FriBidi library

2009-09-28T13:55:19Z

> Date: Mon, 28 Sep 2009 15:58:12 -0400
> From: Behdad Esfahbod <behdad@...>
> CC: fribidi@...
>
> I'm not sure I understand the issue you are raising completely. However, I
> think it's a non-issue since BN characters are conceptually removed in X9, so
> it doesn't matter.

It does matter in an implementation that retains the format codes
instead of removing them, according to section 5.2 of UAX#9.

> However, there is a change around these rules coming in
> the next update to UAX#9 that explicitly supports the code as is, see:
>
> http://www.unicode.org/reports/tr9/tr9-20.html
>
> Does that address your concern?

Yes, thank you.
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Implementation of X6 in the FriBidi library

2009-09-28T12:58:12Z

Hi Eli,

I'm not sure I understand the issue you are raising completely. However, I
think it's a non-issue since BN characters are conceptually removed in X9, so
it doesn't matter. However, there is a change around these rules coming in
the next update to UAX#9 that explicitly supports the code as is, see:

http://www.unicode.org/reports/tr9/tr9-20.html

Does that address your concern?

behdad

On 09/28/2009 01:17 PM, Eli Zaretskii wrote:

> Hi,
>
> Sorry if this is a FAQ, and please point me at it if it is.
>
> The file fribidi-bidi.c in the FriBidi library has this code:
>
> for_run_list (pp, main_run_list)
> {
> FriBidiCharType this_type = RL_TYPE (pp);
> if (FRIBIDI_IS_EXPLICIT_OR_BN (this_type))
> {
> ...
> }
> else
> {
> /* X6. For all types besides RLE, LRE, RLO, LRO, and PDF:
> a. Set the level of the current character to the current
> embedding level.
> b. Whenever the directional override status is not neutral,
> reset the current character type to the directional override
> status. */
> RL_LEVEL (pp) = level;
> if (!FRIBIDI_IS_NEUTRAL (override))
> RL_TYPE (pp) = override;
> }
>
> Unless I'm misreading the code, this means that X6 is not applied to
> characters whose original type is "WEAK BN". However, UAX#9 does not
> exempt WEAK BN characters from the override, so this looks like being
> against the UAX#9 rules.
>
> Am I missing something? are there good reasons for behaving like that?
>
> TIA
> _______________________________________________
> fribidi mailing list
> fribidi@...
> http://lists.freedesktop.org/mailman/listinfo/fribidi
>

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Implementation of X6 in the FriBidi library

2009-09-28T10:15:27Z

Hi,

Sorry if this is a FAQ, and please point me at it if it is.

The file fribidi-bidi.c in the FriBidi library has this code:

for_run_list (pp, main_run_list)
{
FriBidiCharType this_type = RL_TYPE (pp);
if (FRIBIDI_IS_EXPLICIT_OR_BN (this_type))
{
...
}
else
{
/* X6. For all types besides RLE, LRE, RLO, LRO, and PDF:
a. Set the level of the current character to the current
embedding level.
b. Whenever the directional override status is not neutral,
reset the current character type to the directional override
status. */
RL_LEVEL (pp) = level;
if (!FRIBIDI_IS_NEUTRAL (override))
RL_TYPE (pp) = override;
}

Unless I'm misreading the code, this means that X6 is not applied to
characters whose original type is "WEAK BN". However, UAX#9 does not
exempt WEAK BN characters from the override, so this looks like being
against the UAX#9 rules.

Am I missing something? are there good reasons for behaving like that?

TIA
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-23T09:26:22Z

أحمد المحمودي wrote:

Hello,

Are you looking for the string size in characters or in bytes?

---end quoted text---

  Anyone of them is fine.

Then allocate enough storage (character count * 4 ought to be enough), do the reordering + shaping, and then use strlen. That will give you number of bytes.

Yes, if you need precise storage, you may need to copy the string again after that, but there is little that can be done about that. There is no way to know how much memory will be needed without actually performing the operations, so the only way to do this without temporary storage is to do it twice. This, typically, costs more than using temporary storage.

Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting Ltd.
http://www.lingnu.com

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-23T08:04:09Z

Hello,

On Tue, Jun 23, 2009 at 05:40:03PM +0300, Shachar Shemesh wrote:

> >>Also, "sizeof(us)" doesn't return what you meant.
> >
> > Yes, I noticed that it returns the size of pointer type. Well,
> >what I need is to know the size of the string after log2vis,
> >because for example if I got the 2 characters ل & ا after each
> >other, they get translated to a single character, hence the
> >string size decreases. So how do I get the resulting string size
> >?
> >
> Are you looking for the string size in characters or in bytes?

---end quoted text---

Anyone of them is fine.

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-23T07:40:03Z

أحمد المحمودي wrote:

Hello,

On Mon, Jun 22, 2009 at 03:34:41PM -0400, Behdad Esfahbod wrote:

You are not allocating enough for outstr which is UTF8.  Try
allocating 4 bytes per character. That is, (size * 4).


  Thanks, but please notice that instr is also UTF8, and although it 
  consists of 13 characters, yet its size, as returned by sizeof(instr), 
  is 26 !

  Anyways I tried your method, yet I still get the problem I mentioned 
  with: 

  base = FRIBIDI_PAR_RTL;

  That is printf(outstr); shows nothing !

Also, "sizeof(us)" doesn't return what you meant.


  Yes, I noticed that it returns the size of pointer type. Well, what I 
  need is to know the size of the string after log2vis, because for 
  example if I got the 2 characters ل & ا after each other, they get 
  translated to a single character, hence the string size decreases. So 
  how do I get the resulting string size ?

Are you looking for the string size in characters or in bytes?

Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting Ltd.
http://www.lingnu.com

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-23T07:15:46Z

Hello,

On Mon, Jun 22, 2009 at 03:34:41PM -0400, Behdad Esfahbod wrote:
> You are not allocating enough for outstr which is UTF8. Try
> allocating 4 bytes per character. That is, (size * 4).

Thanks, but please notice that instr is also UTF8, and although it
consists of 13 characters, yet its size, as returned by sizeof(instr),
is 26 !

Anyways I tried your method, yet I still get the problem I mentioned
with:

base = FRIBIDI_PAR_RTL;

That is printf(outstr); shows nothing !

> Also, "sizeof(us)" doesn't return what you meant.

Yes, I noticed that it returns the size of pointer type. Well, what I
need is to know the size of the string after log2vis, because for
example if I got the 2 characters ل & ا after each other, they get
translated to a single character, hence the string size decreases. So
how do I get the resulting string size ?

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-22T12:34:41Z

You are not allocating enough for outstr which is UTF8. Try allocating 4
bytes per character. That is, (size * 4). Also, "sizeof(us)" doesn't return
what you meant.

behdad

On 06/22/2009 02:36 PM, أحمد المحمودي wrote:

> On Mon, Jun 22, 2009 at 02:10:35PM -0400, Behdad Esfahbod wrote:
>> Attachment?
>>
>> On 06/22/2009 02:08 PM, أحمد المحمودي wrote:
>>> On Mon, Jun 22, 2009 at 01:32:00PM -0400, Behdad Esfahbod wrote:
>>>> I may remove the obsolete notice in the future. If your paragraph
>>>> is one line of text, you can continue using that function,
>>> ---end quoted text---
>>>
>>> Thanks for the info.
>>>
>>> I have made a small program to test if I understand fribidi right, it is
>>> attached with this email.
>>>
>>> This program works fine as long as: base = FRIBIDI_PAR_LTR;
>>>
>>> Yet if I set: base = FRIBIDI_PAR_RTL;
>>>
>>> I don't get anything printed in the printf statement.
>>>
> ---end quoted text---
>

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-22T11:36:45Z

On Mon, Jun 22, 2009 at 02:10:35PM -0400, Behdad Esfahbod wrote:

> Attachment?
>
> On 06/22/2009 02:08 PM, أحمد المحمودي wrote:
> >On Mon, Jun 22, 2009 at 01:32:00PM -0400, Behdad Esfahbod wrote:
> >>I may remove the obsolete notice in the future. If your paragraph
> >>is one line of text, you can continue using that function,
> >---end quoted text---
> >
> >Thanks for the info.
> >
> >I have made a small program to test if I understand fribidi right, it is
> >attached with this email.
> >
> >This program works fine as long as: base = FRIBIDI_PAR_LTR;
> >
> >Yet if I set: base = FRIBIDI_PAR_RTL;
> >
> >I don't get anything printed in the printf statement.
> >

---end quoted text---

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

#include <stdio.h> #include <stdlib.h> #include <fribidi/fribidi.h> void main(void) { char instr[]="السلام عليكم bro"; FriBidiChar *us, *out_us; char *outstr; //FriBidiCharType base; FriBidiParType base; int size; size=sizeof(instr); us=(FriBidiChar *) malloc(size*sizeof(FriBidiChar)); out_us=(FriBidiChar *) malloc(size*sizeof(FriBidiChar)); outstr=(char *) malloc(size*sizeof(char)); //base = FRIBIDI_TYPE_N; base = FRIBIDI_PAR_LTR; fribidi_charset_to_unicode(FRIBIDI_CHAR_SET_UTF8, instr, size, us); fribidi_log2vis(us, size, &base, out_us, 0, 0, 0); fribidi_unicode_to_charset(FRIBIDI_CHAR_SET_UTF8, out_us, size, outstr); printf("%s\n", outstr); printf("%d %d\n", size, sizeof(us)); free(us); free(out_us); //free(outstr); }

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-22T11:10:35Z

Attachment?

On 06/22/2009 02:08 PM, أحمد المحمودي wrote:

> On Mon, Jun 22, 2009 at 01:32:00PM -0400, Behdad Esfahbod wrote:
>> I may remove the obsolete notice in the future. If your paragraph
>> is one line of text, you can continue using that function,
> ---end quoted text---
>
> Thanks for the info.
>
> I have made a small program to test if I understand fribidi right, it is
> attached with this email.
>
> This program works fine as long as: base = FRIBIDI_PAR_LTR;
>
> Yet if I set: base = FRIBIDI_PAR_RTL;
>
> I don't get anything printed in the printf statement.
>

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-22T11:08:55Z

On Mon, Jun 22, 2009 at 01:32:00PM -0400, Behdad Esfahbod wrote:
> I may remove the obsolete notice in the future. If your paragraph
> is one line of text, you can continue using that function,
---end quoted text---

Thanks for the info.

I have made a small program to test if I understand fribidi right, it is
attached with this email.

This program works fine as long as: base = FRIBIDI_PAR_LTR;

Yet if I set: base = FRIBIDI_PAR_RTL;

I don't get anything printed in the printf statement.

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: fribidi_log2vis changed in 0.19.x ?

2009-06-22T11:03:49Z

They are compatible though.

On 06/22/2009 01:59 PM, أحمد المحمودي wrote:
> Hello,
>
> I noticed a difference between 0.10.x& 0.19.x in fribidi_log2vis
> declaration:
>
> The third argument for this function in 0.10.x is of type
> 'FriBidiCharType *'. While in 0.19.x, the third argument is of type
> 'FriBidiParType *'.
>
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

fribidi_log2vis changed in 0.19.x ?

2009-06-22T10:59:19Z

Hello,

I noticed a difference between 0.10.x & 0.19.x in fribidi_log2vis
declaration:

The third argument for this function in 0.10.x is of type
'FriBidiCharType *'. While in 0.19.x, the third argument is of type
'FriBidiParType *'.

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Obsolete fribidi_log2vis

2009-06-22T10:32:00Z

I may remove the obsolete notice in the future. If your paragraph is one line
of text, you can continue using that function, otherwise, read how it is
implemented and that's what you can do. Yes, it's calling four or five
functions instead, but that's what you have to do if shaping multi-line
paragraphs.

behdad

On 06/22/2009 06:30 AM, أحمد المحمودي wrote:
> Hello,
>
> I'm using fribidi 0.19.2, I was looking at the manpage of
> fribidi_log2vis, and it said that this function is now obsolete, and
> advises to use "other functions" instead. The question is, what
> functions should be used instead ?
>
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: character joining and shaping

2009-06-22T09:58:12Z

Fribidi 0.10.x does NOT support shaping. You need fribidi 0.19.x.

behdad

On 06/22/2009 12:07 PM, Sencer Sultanoglu wrote:

> Hi,
> I've compiled fribidi 0.10.9 on Fedora 9. I read in the list that it
> supports character joining and shaping, I use utf-8, and see joined
> characters using printf on the console. But my application cannot
> display joined characters. I downloaded font ( bdavat ) from :
> http://www.bornaray.com/en_downloads.asp?fn=per_downloads&rfn=en_downloads
> <http://www.bornaray.com/en_downloads.asp?fn=per_downloads&rfn=en_downloads>
> and I converted it to bdf format in order to use in my application. ttf
> to bdf conversion seems to be correct, for example : 0x0621 - 0x064A and
> 0xFE81 - 0xFEFC contains Arabic characters.
> I've also attached samples from the application.
> What might be the problem?
> Thank you,
> Sencer
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> fribidi mailing list
> fribidi@...
> http://lists.freedesktop.org/mailman/listinfo/fribidi

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: character joining and shaping

2009-06-22T09:38:06Z

This is not really answering your question, but I believe that shaping is too difficult to do without a good engine, and even though fribidi can take care of the character order and the glyph substitution, there is much more than that to get good looking output, as well as selections, cursor hit detection, etc. And that's what you get when you use a good layout engine like Pango with or without Gtk or Qt. Unless you have some old application that you want to translate, you should move to a modern engine.

Just my ¢2...

Dov

2009/6/22 Sencer Sultanoglu <sencer.sultanoglu@...>

Hi,

I've compiled fribidi 0.10.9 on Fedora 9. I read in the list that it supports character joining and shaping, I use utf-8, and see joined characters using printf on the console. But my application cannot display joined characters. I downloaded font ( bdavat ) from :

http://www.bornaray.com/en_downloads.asp?fn=per_downloads&rfn=en_downloads

and I converted it to bdf format in order to use in my application. ttf to bdf conversion seems to be correct, for example : 0x0621 - 0x064A and 0xFE81 - 0xFEFC contains Arabic characters.

I've also attached samples from the application.

What might be the problem?

Thank you,

Sencer

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

character joining and shaping

2009-06-22T09:07:47Z

Hi,

I've compiled fribidi 0.10.9 on Fedora 9. I read in the list that it supports character joining and shaping, I use utf-8, and see joined characters using printf on the console. But my application cannot display joined characters. I downloaded font ( bdavat ) from :

http://www.bornaray.com/en_downloads.asp?fn=per_downloads&rfn=en_downloads

and I converted it to bdf format in order to use in my application. ttf to bdf conversion seems to be correct, for example : 0x0621 - 0x064A and 0xFE81 - 0xFEFC contains Arabic characters.

I've also attached samples from the application.

What might be the problem?

Thank you,

Sencer

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

example.PNG (2K) Download Attachment
example2.PNG (3K) Download Attachment

Obsolete fribidi_log2vis

2009-06-22T03:30:46Z

Hello,

I'm using fribidi 0.19.2, I was looking at the manpage of
fribidi_log2vis, and it said that this function is now obsolete, and
advises to use "other functions" instead. The question is, what
functions should be used instead ?

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Mattias Karlsson is out of the office.

2009-05-11T12:02:55Z

I will be out of the office starting 2009-04-10 and will be back
2009-09-01.

I'm on parental leave and will be back in September. I will read my mail
but don't expect an answer back as quick as normal.
/Mattias

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Compiling fribidi-0.19.2 in MinGW/MSYS environment, fix to benchmark.c file

2009-05-11T00:00:47Z

(winXP, msys-1.0.11, mingw_runtime-3.15.2, w32api-3.13-mingw32-dev, gcc-4.2.1-sjlj)
Hello, I was trying to compile fribidi-0.19.2 source in MinGW/MSYS environment. This environment does not have a sys/times.h header file out of the box.

Step 1 is what probably could go to the official source code. I am not sure about step2 if it created any side effects on compiling.

1) fix to utime(void) method
bin/fribidi-benchmark.c file has a minor bug, it uses a a conditional #if HAVE_SYS_TIMES_H+0 tag in import and utime(void) method.

Able to compile one must move "struct tms tb;" row inside #if...#endif block, initial sys/times.h was not enabled so struct is not found.

2) lib/Makefile gives "two or more extra-symbols are not allowed" error message. I had to edit makefile and clear am__append_1 variable then went ok.

See here my steps to compile fribidi package in MinGW/MSYS console.
fribidi-0.19.2.tar.gz
$ tar xvfz zipfile
$ cd libfolder
$ ./configure --prefix=/mingw
edit lib/Makefile and comment extra export-symbols argument
am__append_1 =
##am__append_1 = -export-symbols $(srcdir)/fribidi.def
edit bin/fribidi-benchmark.c, method utime(void), move "struct tms tb;" inside #if....#else block
static double utime (void) {
#if HAVE_SYS_TIMES_H+0
struct tms tb;
times (&tb);
return 0.01 * tb.tms_utime;
#else
#warning Please fill in here to use other functions for determining time.
#endif
}
$ make
$ make install

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Question about symbols & API/ABI compatability

2009-03-30T11:45:11Z

On 03/29/2009 11:43 AM, أحمد المحمودي wrote:

> On Sat, Mar 28, 2009 at 04:59:31PM -0400, Behdad Esfahbod wrote:
>> On 03/28/2009 05:21 AM, أحمد المحمودي wrote:
>>> Hello,
>>>
>>> Thanks for the help, I got another question: are there any symbols in
>>> the file I sent you last time, that are *NOT* missing (or are added by
>>> 0.19.x) yet they are actually internal detail ?
>> Can you send me a listing of all the symbols you are seeing in 0.19.x?
> ---end quoted text---
>
> New symbols in 0.19.x:
> fribidi_debug_status@Base 0.19.1
> fribidi_get_bidi_type@Base 0.19.1
> fribidi_get_bidi_type_name@Base 0.19.1
> fribidi_get_bidi_types@Base 0.19.1
> fribidi_get_joining_type@Base 0.19.1
> fribidi_get_joining_type_name@Base 0.19.1
> fribidi_get_joining_types@Base 0.19.1
> fribidi_get_par_direction@Base 0.19.1
> fribidi_get_par_embedding_levels@Base 0.19.1
> fribidi_join_arabic@Base 0.19.1
> fribidi_reorder_line@Base 0.19.1
> fribidi_shape@Base 0.19.1
> fribidi_shape_arabic@Base 0.19.1
> fribidi_shape_mirroring@Base 0.19.1
> fribidi_unicode_version@Base 0.19.1

These look good.

> Symbols that still exist from 0.10.9:

> fribidi_cap_rtl_to_unicode@Base 0.10.9
> fribidi_char_set_desc_cap_rtl@Base 0.10.9
> fribidi_cp1255_to_unicode_c@Base 0.10.9
> fribidi_cp1256_to_unicode_c@Base 0.10.9
> fribidi_get_type_internal@Base 0.10.9
> fribidi_iso8859_6_to_unicode_c@Base 0.10.9
> fribidi_iso8859_8_to_unicode_c@Base 0.10.9
> fribidi_unicode_to_cap_rtl@Base 0.10.9
> fribidi_unicode_to_charset@Base 0.10.9
> fribidi_unicode_to_cp1255_c@Base 0.10.9
> fribidi_unicode_to_cp1256_c@Base 0.10.9
> fribidi_unicode_to_iso8859_6_c@Base 0.10.9
> fribidi_unicode_to_iso8859_8_c@Base 0.10.9
> fribidi_unicode_to_utf8@Base 0.10.9
> fribidi_utf8_to_unicode@Base 0.10.9

These are all internal symbols. The headers for these are not installed.
When I was writing this code (~2003) I didn't know much about symbol hiding, etc.

Do people think I should go ahead and hide them now? An alternative would be
to just leave them there as I don't see them changing.

behdad

> fribidi_char_set_desc@Base 0.10.9
> fribidi_char_set_name@Base 0.10.9
> fribidi_char_set_title@Base 0.10.9
> fribidi_charset_to_unicode@Base 0.10.9
> fribidi_get_mirror_char@Base 0.10.9
> fribidi_get_type@Base 0.10.9
> fribidi_log2vis@Base 0.10.9
> fribidi_log2vis_get_embedding_levels@Base 0.10.9
> fribidi_mirroring_status@Base 0.10.9
> fribidi_parse_charset@Base 0.10.9
> fribidi_remove_bidi_marks@Base 0.10.9
> fribidi_reorder_nsm_status@Base 0.10.9
> fribidi_set_debug@Base 0.10.9
> fribidi_set_mirroring@Base 0.10.9
> fribidi_set_reorder_nsm@Base 0.10.9
> fribidi_version_info@Base 0.10.9
>

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Question about symbols & API/ABI compatability

2009-03-29T08:43:04Z

On Sat, Mar 28, 2009 at 04:59:31PM -0400, Behdad Esfahbod wrote:
> On 03/28/2009 05:21 AM, أحمد المحمودي wrote:
>> Hello,
>>
>> Thanks for the help, I got another question: are there any symbols in
>> the file I sent you last time, that are *NOT* missing (or are added by
>> 0.19.x) yet they are actually internal detail ?
>
> Can you send me a listing of all the symbols you are seeing in 0.19.x?
---end quoted text---

New symbols in 0.19.x:
fribidi_debug_status@Base 0.19.1
fribidi_get_bidi_type@Base 0.19.1
fribidi_get_bidi_type_name@Base 0.19.1
fribidi_get_bidi_types@Base 0.19.1
fribidi_get_joining_type@Base 0.19.1
fribidi_get_joining_type_name@Base 0.19.1
fribidi_get_joining_types@Base 0.19.1
fribidi_get_par_direction@Base 0.19.1
fribidi_get_par_embedding_levels@Base 0.19.1
fribidi_join_arabic@Base 0.19.1
fribidi_reorder_line@Base 0.19.1
fribidi_shape@Base 0.19.1
fribidi_shape_arabic@Base 0.19.1
fribidi_shape_mirroring@Base 0.19.1
fribidi_unicode_version@Base 0.19.1

Symbols that still exist from 0.10.9:
fribidi_cap_rtl_to_unicode@Base 0.10.9
fribidi_char_set_desc@Base 0.10.9
fribidi_char_set_desc_cap_rtl@Base 0.10.9
fribidi_char_set_name@Base 0.10.9
fribidi_char_set_title@Base 0.10.9
fribidi_charset_to_unicode@Base 0.10.9
fribidi_cp1255_to_unicode_c@Base 0.10.9
fribidi_cp1256_to_unicode_c@Base 0.10.9
fribidi_get_mirror_char@Base 0.10.9
fribidi_get_type@Base 0.10.9
fribidi_get_type_internal@Base 0.10.9
fribidi_iso8859_6_to_unicode_c@Base 0.10.9
fribidi_iso8859_8_to_unicode_c@Base 0.10.9
fribidi_log2vis@Base 0.10.9
fribidi_log2vis_get_embedding_levels@Base 0.10.9
fribidi_mirroring_status@Base 0.10.9
fribidi_parse_charset@Base 0.10.9
fribidi_remove_bidi_marks@Base 0.10.9
fribidi_reorder_nsm_status@Base 0.10.9
fribidi_set_debug@Base 0.10.9
fribidi_set_mirroring@Base 0.10.9
fribidi_set_reorder_nsm@Base 0.10.9
fribidi_unicode_to_cap_rtl@Base 0.10.9
fribidi_unicode_to_charset@Base 0.10.9
fribidi_unicode_to_cp1255_c@Base 0.10.9
fribidi_unicode_to_cp1256_c@Base 0.10.9
fribidi_unicode_to_iso8859_6_c@Base 0.10.9
fribidi_unicode_to_iso8859_8_c@Base 0.10.9
fribidi_unicode_to_utf8@Base 0.10.9
fribidi_utf8_to_unicode@Base 0.10.9
fribidi_version_info@Base 0.10.9

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0x9DCA0B27 (@ subkeys.pgp.net)
GPG Fingerprint: 087D 3767 8CAC 65B1 8F6C 156E D325 C3C8 9DCA 0B27
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Question about symbols & API/ABI compatability

2009-03-28T13:59:31Z

On 03/28/2009 05:21 AM, أحمد المحمودي wrote:
> Hello,
>
> Thanks for the help, I got another question: are there any symbols in
> the file I sent you last time, that are *NOT* missing (or are added by
> 0.19.x) yet they are actually internal detail ?

Can you send me a listing of all the symbols you are seeing in 0.19.x?

behdad
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Question about symbols & API/ABI compatability

2009-03-28T02:21:35Z

Hello,

Thanks for the help, I got another question: are there any symbols in
the file I sent you last time, that are *NOT* missing (or are added by
0.19.x) yet they are actually internal detail ?

--
‎أحمد المحمودي (Ahmed El-Mahmoudy)
Digital design engineer
GPG KeyID: 0x9DCA0B27 (@ subkeys.pgp.net)
GPG Fingerprint: 087D 3767 8CAC 65B1 8F6C 156E D325 C3C8 9DCA 0B27
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

GNU FriBidi 0.19.2 is now available

2009-03-26T15:44:55Z

At the usual location:

http://fribidi.org/download

Overview of changes between 0.19.1 and 0.19.2
=============================================
* Update to Unicode Character Database 5.1.0
* Fixed bug in Arabic ligature table (bug #208870)
* Handle RLM/LRM in CP1255 charset converter. (bug #15328, Artyom)

behdad
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-11T12:59:21Z

Got it. Thanks for the explanation. :)

On Wed, Mar 11, 2009 at 5:31 PM, Behdad Esfahbod <behdad@...> wrote:

> On 03/10/2009 06:08 AM, Behnam Esfahbod ZWNJ wrote:
>
>> Seems the BOM mark appears in the place of the first character of the
>> LAM+ALEF ligature. This might be a but in the ligature replacement of
>> the shaping function.
>>
>> Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
>> fribidi_shape_arabic_ligature(), which replaces the first char of
>> ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.
>
> That's expected. It will be removed by fribidi_remove_bidi_marks().
>
>> Behdad, why don't you use U+FFFF for this purpose?
>
> Because U+FFFF is not a valid character. U+FEFF is harmless at that
> position. And has a BN bidi category and is one of the characters that any
> rendering pipeline removes anyway.
>
>> and why
>> fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
>> clean this CHAR_FILLs?
>
> The shaping functions do not remove any characters. If they did it would be
> hard to keep the mapping between visual and logical strings. The idea is
> that we add filler chars when needing to remove any characters, and
> fribidi_remove_bidi_marks() or similar functions will remove the fillers
> later.
>
>> Behdad, I can fix the problem in CVS if you tell me what's the best
>> way to fix this.
>
> No fix needed AFAI'm concerned :).
>
> behdad
>
>> -Behnam
>>
>>
>

--
' بهنام اسفهبد
' Behnam Esfahbod
'
* .. http://behnam.esfahbod.info
* ` *
* o * http://zwnj.org
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-11T07:50:29Z

On 03/11/2009 10:41 AM, Yoann Roman wrote:

>>> Seems the BOM mark appears in the place of the first character of the
>>> LAM+ALEF ligature. This might be a but in the ligature replacement
>>> of the shaping function.
>>>
>>> Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
>>> fribidi_shape_arabic_ligature(), which replaces the first char of
>>> ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.
>> That's expected. It will be removed by fribidi_remove_bidi_marks().
>>
>>> Behdad, why don't you use U+FFFF for this purpose?
>> Because U+FFFF is not a valid character. U+FEFF is harmless at that
>> position. And has a BN bidi category and is one of the characters
>> that any rendering pipeline removes anyway.
>>
>>> and why
>>> fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
>>> clean this CHAR_FILLs?
>> The shaping functions do not remove any characters. If they did it
>> would be hard to keep the mapping between visual and logical strings.
>> The idea is that we add filler chars when needing to remove any
>> characters, and fribidi_remove_bidi_marks() or similar functions will
>> remove the fillers later.
>>
>>> Behdad, I can fix the problem in CVS if you tell me what's the best
>>> way to fix this.
>> No fix needed AFAI'm concerned :).
>
> So, practically, I need to call fribidi_remove_bidi_marks after calling
> log2vis? I see that's done by fribidi.exe if I pass in --clean.

Yes.

behdad

> Thanks,
>
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-11T07:41:16Z

>> Seems the BOM mark appears in the place of the first character of the
>> LAM+ALEF ligature. This might be a but in the ligature replacement
>> of the shaping function.
>>
>> Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
>> fribidi_shape_arabic_ligature(), which replaces the first char of
>> ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.
>
> That's expected. It will be removed by fribidi_remove_bidi_marks().
>
>> Behdad, why don't you use U+FFFF for this purpose?
>
> Because U+FFFF is not a valid character. U+FEFF is harmless at that
> position. And has a BN bidi category and is one of the characters
> that any rendering pipeline removes anyway.
>
>> and why
>> fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
>> clean this CHAR_FILLs?
>
> The shaping functions do not remove any characters. If they did it
> would be hard to keep the mapping between visual and logical strings.
> The idea is that we add filler chars when needing to remove any
> characters, and fribidi_remove_bidi_marks() or similar functions will
> remove the fillers later.
>
>> Behdad, I can fix the problem in CVS if you tell me what's the best
>> way to fix this.
>
> No fix needed AFAI'm concerned :).

So, practically, I need to call fribidi_remove_bidi_marks after calling
log2vis? I see that's done by fribidi.exe if I pass in --clean.

Thanks,

--
Yoann Roman

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-11T07:01:49Z

On 03/10/2009 06:08 AM, Behnam Esfahbod ZWNJ wrote:

> Seems the BOM mark appears in the place of the first character of the
> LAM+ALEF ligature. This might be a but in the ligature replacement of
> the shaping function.
>
> Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
> fribidi_shape_arabic_ligature(), which replaces the first char of
> ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.

That's expected. It will be removed by fribidi_remove_bidi_marks().

> Behdad, why don't you use U+FFFF for this purpose?

Because U+FFFF is not a valid character. U+FEFF is harmless at that position.
And has a BN bidi category and is one of the characters that any rendering
pipeline removes anyway.

> and why
> fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
> clean this CHAR_FILLs?

The shaping functions do not remove any characters. If they did it would be
hard to keep the mapping between visual and logical strings. The idea is that
we add filler chars when needing to remove any characters, and
fribidi_remove_bidi_marks() or similar functions will remove the fillers later.

> Behdad, I can fix the problem in CVS if you tell me what's the best
> way to fix this.

No fix needed AFAI'm concerned :).

behdad

> -Behnam
>
>
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-10T06:20:58Z

>> Behdad, Thanks for the response.
>
> No problam at all, but please note that I'm behNaM, behDaD's brother.

Sorry for the name mishap. I was looking at Behdad's message while
replying to yours.

>> Your output is what I get from fribidi 0.10.9, but that doesn't do
>> Arabic joining. Other than that BOM mark, the Windows output from
>> 0.19.1 is correct and matches what other non-Fribidi-based, bidi
>> programs do.
>
> I didn't notice you have get the shaping on the string too. So the
> windows output is correct, except the BOM mark.
>
> Seems the BOM mark appears in the place of the first character of the
> LAM+ALEF ligature. This might be a but in the ligature replacement of
> the shaping function.
>
> Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
> fribidi_shape_arabic_ligature(), which replaces the first char of
> ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.
>
> Behdad, why don't you use U+FFFF for this purpose? and why
> fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
> clean this CHAR_FILLs?
>
> Behdad, I can fix the problem in CVS if you tell me what's the best
> way to fix this.

Glad to know it's not a problem just on my end. Thanks for the research.

--
Yoann Roman

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-10T06:20:58Z

>> Thanks for the response.
>>
>> Your output is what I get from fribidi 0.10.9, but that doesn't do
>> Arabic joining. Other than that BOM mark, the Windows output from
>> 0.19.1 is correct and matches what other non-Fribidi-based, bidi
>> programs do.
>>
>>
> I should just point out, as an aside, that the Windows joining is font
> aware. If a certain letter needs to have a left/right join, but the
> font does not have the right glyph, then Windows will use (don't
> remember the details) the right only or left only joined glyph.
>
> Not really relevant to the discussion, but I should point out that
> logical only joining is not full proof.

Thanks for the heads up. In this case, I control the font being used as
well, so I'll just make sure it has all the joining glyphs.

Out of curiosity, how is this handled on other platforms?

--
Yoann Roman

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-10T03:08:52Z

Hi Yoann,

> Behdad, Thanks for the response.

No problam at all, but please note that I'm behNaM, behDaD's brother.

> Your output is what I get from fribidi 0.10.9, but that doesn't do
> Arabic joining. Other than that BOM mark, the Windows output from
> 0.19.1 is correct and matches what other non-Fribidi-based, bidi
> programs do.

I didn't notice you have get the shaping on the string too. So the
windows output is correct, except the BOM mark.

Seems the BOM mark appears in the place of the first character of the
LAM+ALEF ligature. This might be a but in the ligature replacement of
the shaping function.

Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
fribidi_shape_arabic_ligature(), which replaces the first char of
ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.

Behdad, why don't you use U+FFFF for this purpose? and why
fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
clean this CHAR_FILLs?

Behdad, I can fix the problem in CVS if you tell me what's the best
way to fix this.

-Behnam

--
' بهنام اسفهبد
' Behnam Esfahbod
'
* .. http://behnam.esfahbod.info
* ` *
* o * http://zwnj.org
_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi

Re: Invalid UTF-8 for Arabic

2009-03-10T01:00:57Z

Yoann Roman wrote:

>
> Behdad,
>
> Thanks for the response.
>
> Your output is what I get from fribidi 0.10.9, but that doesn't do
> Arabic joining. Other than that BOM mark, the Windows output from
> 0.19.1 is correct and matches what other non-Fribidi-based, bidi
> programs do.
>
>

I should just point out, as an aside, that the Windows joining is font
aware. If a certain letter needs to have a left/right join, but the font
does not have the right glyph, then Windows will use (don't remember the
details) the right only or left only joined glyph.

Not really relevant to the discussion, but I should point out that
logical only joining is not full proof.

Shachar

--
Shachar Shemesh
Lingnu Open Source Consulting Ltd.
http://www.lingnu.com

_______________________________________________
fribidi mailing list
fribidi@...
http://lists.freedesktop.org/mailman/listinfo/fribidi