C# marshaling Unicode characters incorrectly

[dhoffer]

I am using SWIG to create a C# wrapper around a C++ method.

The C++ code is defined as:
const std::wstring& get_Creator() const;

In C# I am testing this by passing in some known Unicode characters,
i.e. \u03A9.  Everything is fine in the SWIG generated JNI code until
the call to:

jresult = SWIG_csharp_wstring_callback(result->c_str());

result variable contains \u03A9 but jresult likely does not.  The
debugger doesn't know what jresult is but when this returns to the C#
side:

public string get_Creator() {
    string ret = ModulePINVOKE.get_Creator(swigCPtr);
    return ret;
}

ret is equal to "O" (0x004f) instead of \u03A9.

What is going wrong here?  Is this a bug in SWIG or am I doing something
wrong?

Regards,
-Dave



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[David Piepgrass]

> I am using SWIG to create a C# wrapper around a C++ method.
>
> The C++ code is defined as:
> const std::wstring& get_Creator() const;
>
> In C# I am testing this by passing in some known Unicode characters,
> i.e. \u03A9.  Everything is fine in the SWIG generated JNI code until
> the call to:
>
> jresult = SWIG_csharp_wstring_callback(result->c_str());
>
> result variable contains \u03A9 but jresult likely does not.  The
> debugger doesn't know what jresult is but when this returns to the C#
> side:
>
> public string get_Creator() {
>     string ret = ModulePINVOKE.get_Creator(swigCPtr);
>     return ret;
> }
>
> ret is equal to "O" (0x004f) instead of \u03A9.
>
> What is going wrong here?  Is this a bug in SWIG or am I doing

something
> wrong?

It's a bug in SWIG. SWIG_csharp_wstring_callback marshals the string as
ANSI so all UNICODE characters are truncated (I'm not sure why you got
0x4F instead of 0xA9 though).

I submitted a related bug:

http://sourceforge.net/tracker/index.php?func=detail&aid=1797418&group_i
d=1645&atid=101645

In my personal copy of SWIG I eliminated SWIG_csharp_wstring_callback
and SWIG_csharp_string_callback. Instead I call SysAllocStringLen, which
is more efficient and does not truncate the string at the first null
character.

It's called P/Invoke by the way, instead of JNI.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[wsfulton]

David Piepgrass wrote:

> > I am using SWIG to create a C# wrapper around a C++ method.
> >
> > The C++ code is defined as:
> > const std::wstring& get_Creator() const;
> >
> > In C# I am testing this by passing in some known Unicode characters,
> > i.e. \u03A9.  Everything is fine in the SWIG generated JNI code until
> > the call to:
> >
> > jresult = SWIG_csharp_wstring_callback(result->c_str());
> >
> > result variable contains \u03A9 but jresult likely does not.  The
> > debugger doesn't know what jresult is but when this returns to the C#
> > side:
> >
> > public string get_Creator() {
> >     string ret = ModulePINVOKE.get_Creator(swigCPtr);
> >     return ret;
> > }
> >
> > ret is equal to "O" (0x004f) instead of \u03A9.
> >
> > What is going wrong here?  Is this a bug in SWIG or am I doing
> something
> > wrong?
>
> It's a bug in SWIG. SWIG_csharp_wstring_callback marshals the string as
> ANSI so all UNICODE characters are truncated (I'm not sure why you got
> 0x4F instead of 0xA9 though).
>
> I submitted a related bug:
>
> http://sourceforge.net/tracker/index.php?func=detail&aid=1797418&group_i
> d=1645&atid=101645
>
> In my personal copy of SWIG I eliminated SWIG_csharp_wstring_callback
> and SWIG_csharp_string_callback. Instead I call SysAllocStringLen, which
> is more efficient and does not truncate the string at the first null
> character.
>

Any reason why you choose SysAllocStringLen over the patch you
submitted? SysAllocStringLen is not platform independent, perhaps that
is the only reason.

William

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[David Piepgrass]

> > In my personal copy of SWIG I eliminated
SWIG_csharp_wstring_callback
> > and SWIG_csharp_string_callback. Instead I call SysAllocStringLen,
which
> > is more efficient and does not truncate the string at the first null
> > character.
> >
> Any reason why you choose SysAllocStringLen over the patch you
> submitted? SysAllocStringLen is not platform independent, perhaps that
> is the only reason.

I submitted the simplest patch that could possibly work. At the time I
submitted it, I don't think I knew that the .NET framework used
SysAllocStringLen under the covers. The way SWIG_csharp_wstring_callback
works is "magic"; the allocation mechanism not documented by Microsoft,
AFAIK.

Although SysAllocStringLen is platform-dependent, it was said that using
UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes on
Linux rather than 2. I have no idea what the "right" solution for Linux
is. I'd suggest using SysAllocStringLen until somebody using Mono on
Linux says what needs to be done there.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[dhoffer]

Thanks for the reply; I thought I was going crazy.  And yes it is P/I
not JNI, I live 90% of the time in Java so it just sort of came out.

I'm not clear on the work-around.  I'm using 1.3.33 (but could change).
What should an end user do to fix this?

-Dave

-----Original Message-----
From: David Piepgrass [mailto:dpiepgrass@...]
Sent: Wednesday, March 12, 2008 6:45 PM
To: William S Fulton
Cc: Dave Hoffer; swig-user@...
Subject: RE: [Swig-user] C# marshaling Unicode characters incorrectly


> > In my personal copy of SWIG I eliminated
SWIG_csharp_wstring_callback
> > and SWIG_csharp_string_callback. Instead I call SysAllocStringLen,
which
> > is more efficient and does not truncate the string at the first null
> > character.
> >
> Any reason why you choose SysAllocStringLen over the patch you
> submitted? SysAllocStringLen is not platform independent, perhaps that
> is the only reason.

I submitted the simplest patch that could possibly work. At the time I
submitted it, I don't think I knew that the .NET framework used
SysAllocStringLen under the covers. The way SWIG_csharp_wstring_callback
works is "magic"; the allocation mechanism not documented by Microsoft,
AFAIK.

Although SysAllocStringLen is platform-dependent, it was said that using
UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes on
Linux rather than 2. I have no idea what the "right" solution for Linux
is. I'd suggest using SysAllocStringLen until somebody using Mono on
Linux says what needs to be done there.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[David Piepgrass]

Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and have
lost track of how exactly mine differs from the standard version. :(

David Piepgrass
Software Engineer-in-Training
__________________________________________
 
Mentor Engineering Inc.
10, 2175 - 29th Street NE
Calgary, AB, Canada  T1Y 7H8
 
Ph: [phone] ext. 490  Fax: [phone]
E-mail: dpiepgrass@...
Website: www.mentoreng.com

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 5:26 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Thanks for the reply; I thought I was going crazy.  And yes it is P/I
> not JNI, I live 90% of the time in Java so it just sort of came out.
>
> I'm not clear on the work-around.  I'm using 1.3.33 (but could

change).

> What should an end user do to fix this?
>
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 6:45 PM
> To: William S Fulton
> Cc: Dave Hoffer; swig-user@...
> Subject: RE: [Swig-user] C# marshaling Unicode characters incorrectly
>
>
> > > In my personal copy of SWIG I eliminated
> SWIG_csharp_wstring_callback
> > > and SWIG_csharp_string_callback. Instead I call SysAllocStringLen,
> which
> > > is more efficient and does not truncate the string at the first

null
> > > character.
> > >
> > Any reason why you choose SysAllocStringLen over the patch you
> > submitted? SysAllocStringLen is not platform independent, perhaps
that
> > is the only reason.
>
> I submitted the simplest patch that could possibly work. At the time I
> submitted it, I don't think I knew that the .NET framework used
> SysAllocStringLen under the covers. The way
SWIG_csharp_wstring_callback
> works is "magic"; the allocation mechanism not documented by
Microsoft,
> AFAIK.
>
> Although SysAllocStringLen is platform-dependent, it was said that
using
> UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes
on
> Linux rather than 2. I have no idea what the "right" solution for
Linux
> is. I'd suggest using SysAllocStringLen until somebody using Mono on
> Linux says what needs to be done there.



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[dhoffer]

David,

Could you send me your code usage of SysAllocStringLen and perhaps your
.i files, maybe I can sort through this.  (I am in a bit of a bind now.)

Does anyone else have a work around?  You say you submitted a patch,
could I build the source with the patch applied (I'm on Windows)?

Thanks,
-Dave

-----Original Message-----
From: David Piepgrass [mailto:dpiepgrass@...]
Sent: Wednesday, March 12, 2008 7:36 PM
To: Dave Hoffer; William S Fulton
Cc: swig-user@...
Subject: RE: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
incorrectly

Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and have
lost track of how exactly mine differs from the standard version. :(

David Piepgrass
Software Engineer-in-Training
__________________________________________
 
Mentor Engineering Inc.
10, 2175 - 29th Street NE
Calgary, AB, Canada  T1Y 7H8
 
Ph: [phone] ext. 490  Fax: [phone]
E-mail: dpiepgrass@...
Website: www.mentoreng.com

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 5:26 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Thanks for the reply; I thought I was going crazy.  And yes it is P/I
> not JNI, I live 90% of the time in Java so it just sort of came out.
>
> I'm not clear on the work-around.  I'm using 1.3.33 (but could

change).

> What should an end user do to fix this?
>
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 6:45 PM
> To: William S Fulton
> Cc: Dave Hoffer; swig-user@...
> Subject: RE: [Swig-user] C# marshaling Unicode characters incorrectly
>
>
> > > In my personal copy of SWIG I eliminated
> SWIG_csharp_wstring_callback
> > > and SWIG_csharp_string_callback. Instead I call SysAllocStringLen,
> which
> > > is more efficient and does not truncate the string at the first

null
> > > character.
> > >
> > Any reason why you choose SysAllocStringLen over the patch you
> > submitted? SysAllocStringLen is not platform independent, perhaps
that
> > is the only reason.
>
> I submitted the simplest patch that could possibly work. At the time I
> submitted it, I don't think I knew that the .NET framework used
> SysAllocStringLen under the covers. The way
SWIG_csharp_wstring_callback
> works is "magic"; the allocation mechanism not documented by
Microsoft,
> AFAIK.
>
> Although SysAllocStringLen is platform-dependent, it was said that
using
> UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes
on
> Linux rather than 2. I have no idea what the "right" solution for
Linux
> is. I'd suggest using SysAllocStringLen until somebody using Mono on
> Linux says what needs to be done there.



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[David Piepgrass]

Take a look at the patch, it's very simple and will let you use UNICODE,
just remember you can't use strings with embedded nulls.

I also attached a .i file that contains most (all? I'm not sure) of my
string-related customizations to SWIG.

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 6:15 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [SPAM] RE: [Swig-user] C# marshaling Unicode
> characters incorrectly
>
> David,
>
> Could you send me your code usage of SysAllocStringLen and perhaps

your
> .i files, maybe I can sort through this.  (I am in a bit of a bind

now.)

>
> Does anyone else have a work around?  You say you submitted a patch,
> could I build the source with the patch applied (I'm on Windows)?
>
> Thanks,
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 7:36 PM
> To: Dave Hoffer; William S Fulton
> Cc: swig-user@...
> Subject: RE: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and

have

> lost track of how exactly mine differs from the standard version. :(
>
> David Piepgrass
> Software Engineer-in-Training
> __________________________________________
>
> Mentor Engineering Inc.
> 10, 2175 - 29th Street NE
> Calgary, AB, Canada  T1Y 7H8
>
> Ph: [phone] ext. 490  Fax: [phone]
> E-mail: dpiepgrass@...
> Website: www.mentoreng.com
>
> > -----Original Message-----
> > From: Dave Hoffer [mailto:DHoffer@...]
> > Sent: Wednesday, March 12, 2008 5:26 PM
> > To: David Piepgrass; William S Fulton
> > Cc: swig-user@...
> > Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> > incorrectly
> >
> > Thanks for the reply; I thought I was going crazy.  And yes it is

P/I

> > not JNI, I live 90% of the time in Java so it just sort of came out.
> >
> > I'm not clear on the work-around.  I'm using 1.3.33 (but could
> change).
> > What should an end user do to fix this?
> >
> > -Dave
> >
> > -----Original Message-----
> > From: David Piepgrass [mailto:dpiepgrass@...]
> > Sent: Wednesday, March 12, 2008 6:45 PM
> > To: William S Fulton
> > Cc: Dave Hoffer; swig-user@...
> > Subject: RE: [Swig-user] C# marshaling Unicode characters

incorrectly
> >
> >
> > > > In my personal copy of SWIG I eliminated
> > SWIG_csharp_wstring_callback
> > > > and SWIG_csharp_string_callback. Instead I call

SysAllocStringLen,

> > which
> > > > is more efficient and does not truncate the string at the first
> null
> > > > character.
> > > >
> > > Any reason why you choose SysAllocStringLen over the patch you
> > > submitted? SysAllocStringLen is not platform independent, perhaps
> that
> > > is the only reason.
> >
> > I submitted the simplest patch that could possibly work. At the time

I

> > submitted it, I don't think I knew that the .NET framework used
> > SysAllocStringLen under the covers. The way
> SWIG_csharp_wstring_callback
> > works is "magic"; the allocation mechanism not documented by
> Microsoft,
> > AFAIK.
> >
> > Although SysAllocStringLen is platform-dependent, it was said that
> using
> > UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes
> on
> > Linux rather than 2. I have no idea what the "right" solution for
> Linux
> > is. I'd suggest using SysAllocStringLen until somebody using Mono on
> > Linux says what needs to be done there.
>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[Stephane Routelous-2]

Hi,

I'm following the discussion about SWIG and strings in C# because last
week we got instabilities ( random, hard to reproduce, memory crashes )
in our application and after reading the posts, I suspect the string
handling of swig to be the reason of the crashes.
I tried to apply the custom_strings.i (I'm using std::string) but it
does not work because I'm using strings in directors and the directors
are not modified to use the SysAllocString but use the old
SWIG_csharp_string_callback mechanism.

If you have any news, or if you need some help in testing, let me
know.

Thanks,

Stephane




Stephane Routelous
stephane.routelous@...
Senior Software Specialist
Rowan Williams Davies & Irwin Inc. (RWDI)
Consulting Engineers & Scientists

650 Woodlawn Road West
Guelph, Ontario, Canada N1K 1B8
T [phone] x2639
F [phone]
www.rwdi.com

>>> "David Piepgrass" <dpiepgrass@...> 13/03/2008 12:38 >>>
Take a look at the patch, it's very simple and will let you use
UNICODE,
just remember you can't use strings with embedded nulls.

I also attached a .i file that contains most (all? I'm not sure) of my
string-related customizations to SWIG.

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 6:15 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [SPAM] RE: [Swig-user] C# marshaling Unicode
> characters incorrectly
>
> David,
>
> Could you send me your code usage of SysAllocStringLen and perhaps

your
> .i files, maybe I can sort through this.  (I am in a bit of a bind
now.)

>
> Does anyone else have a work around?  You say you submitted a patch,
> could I build the source with the patch applied (I'm on Windows)?
>
> Thanks,
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 7:36 PM
> To: Dave Hoffer; William S Fulton
> Cc: swig-user@...
> Subject: RE: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and

have

> lost track of how exactly mine differs from the standard version. :(
>
> David Piepgrass
> Software Engineer-in-Training
> __________________________________________
>
> Mentor Engineering Inc.
> 10, 2175 - 29th Street NE
> Calgary, AB, Canada  T1Y 7H8
>
> Ph: [phone] ext. 490  Fax: [phone]
> E-mail: dpiepgrass@...
> Website: www.mentoreng.com
>
> > -----Original Message-----
> > From: Dave Hoffer [mailto:DHoffer@...]
> > Sent: Wednesday, March 12, 2008 5:26 PM
> > To: David Piepgrass; William S Fulton
> > Cc: swig-user@...
> > Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> > incorrectly
> >
> > Thanks for the reply; I thought I was going crazy.  And yes it is

P/I
> > not JNI, I live 90% of the time in Java so it just sort of came
out.

> >
> > I'm not clear on the work-around.  I'm using 1.3.33 (but could
> change).
> > What should an end user do to fix this?
> >
> > -Dave
> >
> > -----Original Message-----
> > From: David Piepgrass [mailto:dpiepgrass@...]
> > Sent: Wednesday, March 12, 2008 6:45 PM
> > To: William S Fulton
> > Cc: Dave Hoffer; swig-user@...
> > Subject: RE: [Swig-user] C# marshaling Unicode characters

incorrectly
> >
> >
> > > > In my personal copy of SWIG I eliminated
> > SWIG_csharp_wstring_callback
> > > > and SWIG_csharp_string_callback. Instead I call
SysAllocStringLen,
> > which
> > > > is more efficient and does not truncate the string at the
first
> null
> > > > character.
> > > >
> > > Any reason why you choose SysAllocStringLen over the patch you
> > > submitted? SysAllocStringLen is not platform independent,
perhaps
> that
> > > is the only reason.
> >
> > I submitted the simplest patch that could possibly work. At the
time
I

> > submitted it, I don't think I knew that the .NET framework used
> > SysAllocStringLen under the covers. The way
> SWIG_csharp_wstring_callback
> > works is "magic"; the allocation mechanism not documented by
> Microsoft,
> > AFAIK.
> >
> > Although SysAllocStringLen is platform-dependent, it was said that
> using
> > UnmanagedType.U2 is also platform-dependent since wchar_t is 4

bytes
> on
> > Linux rather than 2. I have no idea what the "right" solution for
> Linux
> > is. I'd suggest using SysAllocStringLen until somebody using Mono
on
> > Linux says what needs to be done there.
>


______________________________________________________________

Celebrating Excellence Since 1972. For more information, please visit www.rwdi.com/35th_anniversary/.

Reputation Resources Results

http://www.rwdi.com 

______________________________________________________________

This communication is intended for the sole use of the party to whom it is addressed and may contain information that is privileged and/or confidential. Any other distribution, copying or disclosure is strictly prohibited. If you have received this e-mail in error, please notify us immediately by telephone and delete the message without retaining any hard or electronic copies of same.

RWDI scans outgoing emails for viruses, but makes no warranty as to their absence in this email or attachments.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[dhoffer]

Okay, thanks I am trying to implement this.  

Do I understand correctly, that there is a swig patch as well as the .i?
How do I get to your patch?

-Dave

-----Original Message-----
From: David Piepgrass [mailto:dpiepgrass@...]
Sent: Thursday, March 13, 2008 12:38 PM
To: Dave Hoffer; William S Fulton
Cc: swig-user@...
Subject: RE: [Swig-user] C# marshaling Unicode characters incorrectly

Take a look at the patch, it's very simple and will let you use UNICODE,
just remember you can't use strings with embedded nulls.

I also attached a .i file that contains most (all? I'm not sure) of my
string-related customizations to SWIG.

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 6:15 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [SPAM] RE: [Swig-user] C# marshaling Unicode
> characters incorrectly
>
> David,
>
> Could you send me your code usage of SysAllocStringLen and perhaps

your
> .i files, maybe I can sort through this.  (I am in a bit of a bind
now.)

>
> Does anyone else have a work around?  You say you submitted a patch,
> could I build the source with the patch applied (I'm on Windows)?
>
> Thanks,
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 7:36 PM
> To: Dave Hoffer; William S Fulton
> Cc: swig-user@...
> Subject: RE: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and

have

> lost track of how exactly mine differs from the standard version. :(
>
> David Piepgrass
> Software Engineer-in-Training
> __________________________________________
>
> Mentor Engineering Inc.
> 10, 2175 - 29th Street NE
> Calgary, AB, Canada  T1Y 7H8
>
> Ph: [phone] ext. 490  Fax: [phone]
> E-mail: dpiepgrass@...
> Website: www.mentoreng.com
>
> > -----Original Message-----
> > From: Dave Hoffer [mailto:DHoffer@...]
> > Sent: Wednesday, March 12, 2008 5:26 PM
> > To: David Piepgrass; William S Fulton
> > Cc: swig-user@...
> > Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> > incorrectly
> >
> > Thanks for the reply; I thought I was going crazy.  And yes it is

P/I

> > not JNI, I live 90% of the time in Java so it just sort of came out.
> >
> > I'm not clear on the work-around.  I'm using 1.3.33 (but could
> change).
> > What should an end user do to fix this?
> >
> > -Dave
> >
> > -----Original Message-----
> > From: David Piepgrass [mailto:dpiepgrass@...]
> > Sent: Wednesday, March 12, 2008 6:45 PM
> > To: William S Fulton
> > Cc: Dave Hoffer; swig-user@...
> > Subject: RE: [Swig-user] C# marshaling Unicode characters

incorrectly
> >
> >
> > > > In my personal copy of SWIG I eliminated
> > SWIG_csharp_wstring_callback
> > > > and SWIG_csharp_string_callback. Instead I call
SysAllocStringLen,

> > which
> > > > is more efficient and does not truncate the string at the first
> null
> > > > character.
> > > >
> > > Any reason why you choose SysAllocStringLen over the patch you
> > > submitted? SysAllocStringLen is not platform independent, perhaps
> that
> > > is the only reason.
> >
> > I submitted the simplest patch that could possibly work. At the time

I

> > submitted it, I don't think I knew that the .NET framework used
> > SysAllocStringLen under the covers. The way
> SWIG_csharp_wstring_callback
> > works is "magic"; the allocation mechanism not documented by
> Microsoft,
> > AFAIK.
> >
> > Although SysAllocStringLen is platform-dependent, it was said that
> using
> > UnmanagedType.U2 is also platform-dependent since wchar_t is 4 bytes
> on
> > Linux rather than 2. I have no idea what the "right" solution for
> Linux
> > is. I'd suggest using SysAllocStringLen until somebody using Mono on
> > Linux says what needs to be done there.
>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[dhoffer]

Sorry, I was able to get mine to work with custom_strings.i so I didn't
need to look into directors.

-Dave

-----Original Message-----
From: Stephane Routelous [mailto:Stephane.Routelous@...]
Sent: Thursday, March 13, 2008 2:03 PM
To: William S Fulton; David Piepgrass; Dave Hoffer
Cc: swig-user@...
Subject: Re: [Swig-user] C# marshaling Unicode characters incorrectly

Hi,

I'm following the discussion about SWIG and strings in C# because last
week we got instabilities ( random, hard to reproduce, memory crashes )
in our application and after reading the posts, I suspect the string
handling of swig to be the reason of the crashes.
I tried to apply the custom_strings.i (I'm using std::string) but it
does not work because I'm using strings in directors and the directors
are not modified to use the SysAllocString but use the old
SWIG_csharp_string_callback mechanism.

If you have any news, or if you need some help in testing, let me
know.

Thanks,

Stephane




Stephane Routelous
stephane.routelous@...
Senior Software Specialist
Rowan Williams Davies & Irwin Inc. (RWDI)
Consulting Engineers & Scientists

650 Woodlawn Road West
Guelph, Ontario, Canada N1K 1B8
T [phone] x2639
F [phone]
www.rwdi.com

>>> "David Piepgrass" <dpiepgrass@...> 13/03/2008 12:38 >>>
Take a look at the patch, it's very simple and will let you use
UNICODE,
just remember you can't use strings with embedded nulls.

I also attached a .i file that contains most (all? I'm not sure) of my
string-related customizations to SWIG.

> -----Original Message-----
> From: Dave Hoffer [mailto:DHoffer@...]
> Sent: Wednesday, March 12, 2008 6:15 PM
> To: David Piepgrass; William S Fulton
> Cc: swig-user@...
> Subject: [SPAM] RE: [SPAM] RE: [Swig-user] C# marshaling Unicode
> characters incorrectly
>
> David,
>
> Could you send me your code usage of SysAllocStringLen and perhaps

your
> .i files, maybe I can sort through this.  (I am in a bit of a bind
now.)

>
> Does anyone else have a work around?  You say you submitted a patch,
> could I build the source with the patch applied (I'm on Windows)?
>
> Thanks,
> -Dave
>
> -----Original Message-----
> From: David Piepgrass [mailto:dpiepgrass@...]
> Sent: Wednesday, March 12, 2008 7:36 PM
> To: Dave Hoffer; William S Fulton
> Cc: swig-user@...
> Subject: RE: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> incorrectly
>
> Sorry, I'm not sure. I customized my SWIG's .i files somewhat, and

have

> lost track of how exactly mine differs from the standard version. :(
>
> David Piepgrass
> Software Engineer-in-Training
> __________________________________________
>
> Mentor Engineering Inc.
> 10, 2175 - 29th Street NE
> Calgary, AB, Canada  T1Y 7H8
>
> Ph: [phone] ext. 490  Fax: [phone]
> E-mail: dpiepgrass@...
> Website: www.mentoreng.com
>
> > -----Original Message-----
> > From: Dave Hoffer [mailto:DHoffer@...]
> > Sent: Wednesday, March 12, 2008 5:26 PM
> > To: David Piepgrass; William S Fulton
> > Cc: swig-user@...
> > Subject: [SPAM] RE: [Swig-user] C# marshaling Unicode characters
> > incorrectly
> >
> > Thanks for the reply; I thought I was going crazy.  And yes it is

P/I
> > not JNI, I live 90% of the time in Java so it just sort of came
out.

> >
> > I'm not clear on the work-around.  I'm using 1.3.33 (but could
> change).
> > What should an end user do to fix this?
> >
> > -Dave
> >
> > -----Original Message-----
> > From: David Piepgrass [mailto:dpiepgrass@...]
> > Sent: Wednesday, March 12, 2008 6:45 PM
> > To: William S Fulton
> > Cc: Dave Hoffer; swig-user@...
> > Subject: RE: [Swig-user] C# marshaling Unicode characters

incorrectly
> >
> >
> > > > In my personal copy of SWIG I eliminated
> > SWIG_csharp_wstring_callback
> > > > and SWIG_csharp_string_callback. Instead I call
SysAllocStringLen,
> > which
> > > > is more efficient and does not truncate the string at the
first
> null
> > > > character.
> > > >
> > > Any reason why you choose SysAllocStringLen over the patch you
> > > submitted? SysAllocStringLen is not platform independent,
perhaps
> that
> > > is the only reason.
> >
> > I submitted the simplest patch that could possibly work. At the
time
I

> > submitted it, I don't think I knew that the .NET framework used
> > SysAllocStringLen under the covers. The way
> SWIG_csharp_wstring_callback
> > works is "magic"; the allocation mechanism not documented by
> Microsoft,
> > AFAIK.
> >
> > Although SysAllocStringLen is platform-dependent, it was said that
> using
> > UnmanagedType.U2 is also platform-dependent since wchar_t is 4

bytes
> on
> > Linux rather than 2. I have no idea what the "right" solution for
> Linux
> > is. I'd suggest using SysAllocStringLen until somebody using Mono
on
> > Linux says what needs to be done there.
>


______________________________________________________________

Celebrating Excellence Since 1972. For more information, please visit
www.rwdi.com/35th_anniversary/.

Reputation Resources Results

http://www.rwdi.com 

______________________________________________________________

This communication is intended for the sole use of the party to whom it
is addressed and may contain information that is privileged and/or
confidential. Any other distribution, copying or disclosure is strictly
prohibited. If you have received this e-mail in error, please notify us
immediately by telephone and delete the message without retaining any
hard or electronic copies of same.

RWDI scans outgoing emails for viruses, but makes no warranty as to
their absence in this email or attachments.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[wsfulton]

David Piepgrass wrote:
> Take a look at the patch, it's very simple and will let you use UNICODE,
> just remember you can't use strings with embedded nulls.
>

I keep seeing mention of strings with embedded nulls, but can someone
explain how one puts a null in a string? How can this work when strings
are null terminated and is fundamental to the way they work?

William

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[David Piepgrass]

C++'s std::string and C#'s System.String contain size/Length fields that
specify the string length. Consequently you are allowed to have embedded
nulls in both. Ditto for Java's string, I think. And std::wstring, of
course.

Although std::string and std::wstring have a null terminator at the end,
that is only for backward compatibility with C. The terminator is
otherwise not counted as part of the string. By the way, I think
Microsoft's BSTR and CString also allow embedded nulls (they are
length-prefixed).

> -----Original Message-----
> David Piepgrass wrote:
> > Take a look at the patch, it's very simple and will let you use
UNICODE,
> > just remember you can't use strings with embedded nulls.
> >
>
> I keep seeing mention of strings with embedded nulls, but can someone
> explain how one puts a null in a string? How can this work when
strings
> are null terminated and is fundamental to the way they work?
>
> William


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[Abhinandan Jain]

Hi

I believe that thread protection code is missing from wrapper code
generated for STL iterators for the Python target language in swig
1.3.34 and prior versions. The missing code can cause the Python
interpreter to crash.

The current swig generated code includes lines should such as the
following for the PySwigIterator C++ class created for std::vector
wrappers:

      PyObject *next()
      {
        PyObject *obj = value();
        incr();
        return obj;
      }

      PyObject *previous()
      {
        decr();
        return value();
      }

The value() method's implementation happens to use Python C API calls
and hence requires thread protection. The following is the correct
code that should be generated:

     PyObject *next()
     {
       SWIG_PYTHON_THREAD_BEGIN_BLOCK; // disable threads
       PyObject *obj = value();
       incr();
       SWIG_PYTHON_THREAD_END_BLOCK; // re-enable threads
       return obj;
     }

     PyObject *previous()
     {
       SWIG_PYTHON_THREAD_BEGIN_BLOCK; // disable threads
       decr();
       // return value();
       PyObject *obj = value();
       SWIG_PYTHON_THREAD_END_BLOCK; // re-enable threads
       return obj;
     }

The above fix needs to be applied to the Lib/python/pyiterators.swg
file in the distribution to fix the auto-code generation. There may be
other such template code that may require such protection in this area
- which someone more knowlegeable would know.

I hope this fix makes it into the next version of SWIG.

Thanks.

Abhi



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user