Unicode mode

[aparajita]

Hi,

I strongly encourage everyone using v11 to use Unicode mode. Non-
Unicode mode is essentially a hack -- internally 4D is always using  
Unicode throughout (including the data in the database). When you run  
in non-Unicode mode you are forcing 4D to constantly convert to and  
from Mac Roman -- every time you read and write data, and every time  
you read and write strings in the language. Essentially you are asking  
4D to cripple itself -- and waste a lot of time doing so.

The real killer is when you pass text/string arrays from a non-Unicode  
database to a Unicode plugin (such as Active4D). 4D has to copy the  
entire array, convert each element, pass that to the plugin, and then  
after calling the plugin do the same in reverse.

Please note that Active4D v11 is ALWAYS in Unicode mode. You may not  
be able to access 4D process or interprocess text/string variables in  
Active4D if the database is in non-Unicode mode. I haven't tested this  
yet.

I sense something approaching fear or scorn in some of the responses.  
Unicode is nothing to fear, in fact it should be embraced. EVERY  
operating system and EVERY major development platform outside of 4D  
uses Unicode, it's about time we in the 4D community catch up (kicking  
and screaming). Unicode may be new, it may be unknown, but it is  
nothing to fear.

Please note: I DO understand that many of you are under time  
constraints and don't feel you have the time to deal with issues that  
*may* come up with the switch to Unicode.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Michael Check]

No fear or scorn here. Just thought that if main purpose was to get  
i18n conversions, we wouldn't care. However this shows my ignorance  
since having essentially no limit text strings and avoiding overhead  
in conversions is certainly better.

So count us in then for converting to Unicode. We do not have a tight  
time constraint so we will do the conversion during initial v11  
migration. I would like to get an idea of what types of hurdles have/
will come up - does anyone have a general sense of this?


Thanks,

Michael Check

On Sep 15, 2009, at 1:44 PM, Aparajita Fishman <aparajita@...
 > wrote:

> Hi,
>
> I strongly encourage everyone using v11 to use Unicode mode. Non-
> Unicode mode is essentially a hack -- internally 4D is always using  
> Unicode throughout (including the data in the database). When you  
> run in non-Unicode mode you are forcing 4D to constantly convert to  
> and from Mac Roman -- every time you read and write data, and every  
> time you read and write strings in the language. Essentially you are  
> asking 4D to cripple itself -- and waste a lot of time doing so.
>
> The real killer is when you pass text/string arrays from a non-
> Unicode database to a Unicode plugin (such as Active4D). 4D has to  
> copy the entire array, convert each element, pass that to the  
> plugin, and then after calling the plugin do the same in reverse.
>
> Please note that Active4D v11 is ALWAYS in Unicode mode. You may not  
> be able to access 4D process or interprocess text/string variables  
> in Active4D if the database is in non-Unicode mode. I haven't tested  
> this yet.
>
> I sense something approaching fear or scorn in some of the  
> responses. Unicode is nothing to fear, in fact it should be  
> embraced. EVERY operating system and EVERY major development  
> platform outside of 4D uses Unicode, it's about time we in the 4D  
> community catch up (kicking and screaming). Unicode may be new, it  
> may be unknown, but it is nothing to fear.
>
> Please note: I DO understand that many of you are under time  
> constraints and don't feel you have the time to deal with issues  
> that *may* come up with the switch to Unicode.
>
> Regards,
>
>   Aparajita
>   www.aparajitaworld.com
>
>   "If you dare to fail, you are bound to succeed."
>   - Sri Chinmoy   |   www.srichinmoy.org
>
> _______________________________________________
> Active4D-dev mailing list
> Active4D-dev@...
> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Alexander Heintz]

Am 15.09.2009 um 20:44 schrieb Aparajita Fishman:

> Please note: I DO understand that many of you are under time  
> constraints and don't feel you have the time to deal with issues  
> that *may* come up with the switch to Unicode.

Believe me, As I have done this a few times:

The ONLY thing to get rid of are all instances of

MacToISO
MacToWin
WinToMac
ISOToMac

and with the global replace function in V11 simply replace all calls  
ith a pass-through wrapper, solving this in seconds!

`PM: ConvertPassThrough
c_text($0;$1)
$0:=$1

Of course if NEED to export ISO, Mac or Win Character Set is is a few  
minutes of work, but nothing in the least troubling.

everything else is absolutely painless!

Take a GERMAN programmers word for it, and we all know that german  
developers are overpriced perfectionists, just like german cars (some  
of the at least) :-)

Cheers

Alex
_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> The ONLY thing to get rid of are all instances of
>
> MacToISO
> MacToWin
> WinToMac
> ISOToMac

No need to get rid of them or wrap them, they are already no-ops in  
Unicode mode.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> having essentially no limit text strings

This whole dichotomy between string and text variables made sense  
maybe 10 years ago when memory and CPU was not as cheap and plentiful  
as it is today. It's a needless diversion nowadays. And the whole idea  
of text being limited to 32000 characters is equally archaic.


> and avoiding overhead in conversions is certainly better.

When it comes to ObjectTools, the overhead is significant. Someone in  
France did testing and found some operations were >100 times slower in  
compatibility mode.


> I would like to get an idea of what types of hurdles have/will come  
> up - does anyone have a general sense of this?

Do a gmane search for "unicode problems". There aren't that many. The  
main ones:

- Don't store control characters (or generic binary data) in strings  
and expect to be able to search on them

- If you need to check if a character is a particular control  
character, you have to compare with its character code

- C_STRING and ARRAY STRING variables don't exist in Unicode mode,  
even though you can declare them. Everything is text with a 2GB length  
limit. So if you were depending on fixed string lengths to clip  
variable lengths, you will have to check/clip the lengths manually.  
Note that this does not apply to alpha database fields, 4D handles the  
length clipping for you as usual.

- The Length() command returns the number of Unicode 16-bit code units  
in a string. This covers the vast majority of cases, since all living  
languages can be encoded within a single 16-bit Unicode code unit. If  
the text requires 2 Unicode 16-bit code units to encode some  
characters, the number returned by Length() will not match the number  
of logical "characters". Even then, unless you are iterating through  
the string character by character, you won't care about this  
discrepancy.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> Please note that Active4D v11 is ALWAYS in Unicode mode. You may not  
> be able to access 4D process or interprocess text/string variables  
> in Active4D if the database is in non-Unicode mode. I haven't tested  
> this yet.

I have tested this now, and it's much better than I thought it would be:

In Unicode mode there are no restrictions on process/interprocess  
variable usage.

In non-Unicode mode:
- Text variables/arrays are read write.
- String variables/arrays appear to Active4D as text variables/arrays.
- If you declare a process/interprocess string variable/array in 4D,  
it is read only.
- If you declare a process/interprocess string variable/array in  
Active4D, it is read write.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Michael Kaye-4]

Aparajita,

I think we are all agreed that unicode mode is best....but I am  
intrigued as to why you asked the question initially.

Is there an advantage to you if Active4D doesn't have to support  
compatibility mode? Will it save you some development time/testing?

If yes I for one would be happy with no compatibility mode (at least  
in the first release) - that way we all benefit from having A4D v11  
sooner.

M.

On 15 Sep 2009, at 20:55, Aparajita Fishman wrote:

>> Please note that Active4D v11 is ALWAYS in Unicode mode. You may  
>> not be able to access 4D process or interprocess text/string  
>> variables in Active4D if the database is in non-Unicode mode. I  
>> haven't tested this yet.
>
> I have tested this now, and it's much better than I thought it would  
> be:
>
> In Unicode mode there are no restrictions on process/interprocess  
> variable usage.
>
> In non-Unicode mode:
> - Text variables/arrays are read write.
> - String variables/arrays appear to Active4D as text variables/arrays.
> - If you declare a process/interprocess string variable/array in 4D,  
> it is read only.
> - If you declare a process/interprocess string variable/array in  
> Active4D, it is read write.
>
> Regards,
>
>   Aparajita
>   www.aparajitaworld.com
>
>   "If you dare to fail, you are bound to succeed."
>   - Sri Chinmoy   |   www.srichinmoy.org
>
> _______________________________________________
> Active4D-dev mailing list
> Active4D-dev@...
> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> I am intrigued as to why you asked the question initially.

I am sure there will be some small issues that arise (such as the read  
only nature of process string variables) with v11 sites that run in  
compatibility mode.


> Is there an advantage to you if Active4D doesn't have to support  
> compatibility mode? Will it save you some development time/testing?

Yes, it is all a matter of saving time at this point. I have spent a  
year(!!) on the v11 version, and frankly I'm ready to cut any corners  
I can.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Alexander Heintz]

>> Is there an advantage to you if Active4D doesn't have to support  
>> compatibility mode? Will it save you some development time/testing?
>
> Yes, it is all a matter of saving time at this point. I have spent a  
> year(!!) on the v11 version, and frankly I'm ready to cut any  
> corners I can.

Be my guest in cutting this one! :-)
_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Peter Schumacher-2]

Aparajita,

just cut the non-unicode mode. Doing web stuff and staying away from  
unicode is like Snow Leopard on MacOS 9.

Am 15.09.2009 um 22:24 schrieb Aparajita Fishman:

> Yes, it is all a matter of saving time at this point. I have spent a  
> year(!!) on the v11 version, and frankly I'm ready to cut any  
> corners I can.




Greetings,
Peter

iChat/Skype: PeterInWiesbaden

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Mike Vogt]

Hi all,
This might not be an Active4D question, but I'll ask anyway.  I have a
fuseaction that set content type("application/vnd.ms-excel") that worked
great, asking the browser user if they wanted to open in excel, etc.
However, last night, we mad the entire site secure, so all urls are forced
to https if they are http, using Apache rewrite in from of 4D Server (we've
always used Apache rewrite, just not for https).  Now, all the excel links
give an error saying 'Could not open 'https://www.willsub.com/index...
My question is, is there a response header or other info I can set so that
the response is set back to http from https?  Is there any other way to look
at this problem?

TIA,
Sincerely,
Mike Vogt
Willsub Technical Support


_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> My question is, is there a response header or other info I can set  
> so that
> the response is set back to http from https?

Not within Active4D. You have no control over the transport mechanism  
within Active4D. You may be able to do something within apache.

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Michael Bond-4]

2009/9/16 Mike Vogt <mv@...>

> Hi all,
> This might not be an Active4D question, but I'll ask anyway.  I have a
> fuseaction that set content type("application/vnd.ms-excel") that worked
> great, asking the browser user if they wanted to open in excel, etc.
> However, last night, we mad the entire site secure, so all urls are forced
> to https if they are http, using Apache rewrite in from of 4D Server (we've
> always used Apache rewrite, just not for https).  Now, all the excel links
> give an error saying 'Could not open 'https://www.willsub.com/index...
> My question is, is there a response header or other info I can set so that
> the response is set back to http from https?  Is there any other way to
> look
> at this problem?
>
> TIA,
> Sincerely,
> Mike Vogt
> Willsub Technical Support
>
> I'll be honest I don't really understand the problem but if you need a

secure request to be "transformed" under a certain rule set to a standard
non-secure then you could simply use a Redirect from within Apache or
Active4D to the non-secure pages with the original request parameters
attached to the Redirect.

Michael Bond

Michael
_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Mike Vogt]

It turns out that IE can't correctly access Office documents from a secure
site!  This problem does not exist in Firefox, Opera, or Safari.  The work
around is that the cache must be enabled for that request.  Adding set cache
control("Cache-Control: max-age=120, must-revalidate") just before the set
content type("application/vnd.ms-excel") fixed the problem.  The following
Microsoft knowledgebase article  documents the problem and the solution.

http://support.microsoft.com/kb/316431

 
Thanks,
Mike

> -----Original Message-----
> From: active4d-dev-bounces@...
> [mailto:active4d-dev-bounces@...] On Behalf Of
> Michael Bond
> Sent: Wednesday, September 16, 2009 9:43 AM
> To: active4d-dev@...
> Subject: Re: [Active4d-dev] HTTPS in HTTP out
>
> 2009/9/16 Mike Vogt <mv@...>
>
> > Hi all,
> > This might not be an Active4D question, but I'll ask
> anyway.  I have a
> > fuseaction that set content type("application/vnd.ms-excel") that
> > worked great, asking the browser user if they wanted to
> open in excel, etc.
> > However, last night, we mad the entire site secure, so all urls are
> > forced to https if they are http, using Apache rewrite in
> from of 4D
> > Server (we've always used Apache rewrite, just not for
> https).  Now,
> > all the excel links give an error saying 'Could not open
> 'https://www.willsub.com/index...
> > My question is, is there a response header or other info I
> can set so
> > that the response is set back to http from https?  Is there
> any other
> > way to look at this problem?
> >
> > TIA,
> > Sincerely,
> > Mike Vogt
> > Willsub Technical Support
> >
> > I'll be honest I don't really understand the problem but if
> you need a
> secure request to be "transformed" under a certain rule set
> to a standard non-secure then you could simply use a Redirect
> from within Apache or Active4D to the non-secure pages with
> the original request parameters attached to the Redirect.
>
> Michael Bond
>
> Michael
> _______________________________________________
> Active4D-dev mailing list
> Active4D-dev@...
> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[aparajita]

> It turns out that IE can't correctly access Office documents from a  
> secure
> site!  This problem does not exist in Firefox, Opera, or Safari.

Imagine that! A problem with IE!!   ;-)

Regards,

    Aparajita
    www.aparajitaworld.com

    "If you dare to fail, you are bound to succeed."
    - Sri Chinmoy   |   www.srichinmoy.org

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/

[Raimon Fernandez Sirera-2]

Hello,

This is the same problem I had some time ago with PDF and SSL and of  
course, with IE  ...

And the solution was:

set cache control("")

* we 'love' IE *


regards,

r.


Begin forwarded message:

> From: Raimon Fernandez <4d@...>
> Date: 23 de febrero de 2005 16:54:15 GMT+01:00
> To: active4d-dev@...
> Subject: [Active4d-dev] PDF over SSL Error with Internet Explorer
> Reply-To: Active4D Developer Discussion List <active4d-dev@...
> >
>
> Hi !
>
> We are serving some pdfs, in normal connection HTTP all runs  
> perfect, but when the connection is secured, HTTPS, our friend IE  
> doesn't want to download the file.
>
> This problems only occurs in windows and with IE, with Opera,  
> Mozilla, ... all works as expected.
>
> After making a google search, the result is awsome, but in resume ...
>
>
> Problem:
> ----------------------
> When opening a PDF over SSL running on IIS 5.1 (Windows XP  
> Professional, W2K Windows 2000, or Windows Server 2003), you may  
> receive the following problem with Internet Explorer.
>
> First, instead of opening the PDF in Acrobat Reader, IE asks you,  
> "Do you want to open or save this file?". If you click "Open", then  
> you receive the error, "Internet Explorer cannot download file.pdf  
> from servername. Internet Explorer was not able to open this  
> Internet site. The requested site is unavailable or cannot be found.  
> Please try again later."
>
> If you right-click on the link to the PDF, and select "Save", you  
> receive the following error: "Internet Explorer cannot download  
> file.pdf from servername. The file could not be written to the cache."
>
>
> Cause:
> ----------------------------------
> This problem occurs because the PDF returned by the web server  
> contains HTTP headers to disable caching. Specifically, the header  
> is the no-cache pragma. In VBScript ASP, the code lines that can  
> cause this error are:
> Response.AddHeader "pragma","no-cache"
> or
> Response.CacheControl = "no-cache"
>
>
> Solution:
> ------------------
> Remove the above headers for disabling caching.
>
>
>
> So, after reading againg the Active4D manual, I think I can do it  
> pretty easy, but ...
>
> This is the response from the web server, with the file:
>
> 192.168.000.154.08080-192.168.000.004.03066: HTTP/1.0 200 OK
> Server: 4D_WebStar_D/7.5
> Cache-Control: no-cache
> Pragma: no-cache
> Expires: Wed, 23 Feb 2005 15:39:47 GMT
> Content-Disposition: attachment; filename=RE001004.pdf
> Content-Type: application/octet-stream
> Date: Wed, 23 Feb 2005 15:39:47 GMT
> Last-Modified: Wed, 23 Feb 2005 15:39:47 GMT
> Content-Type: application/octet-stream
> Content-Length: 51756
>
> For what I read, I think just removing the Cache-Control: no-cache  
> and Pragma: no-cache should work, but ...
>
> Here is the Active4D code ...
>
>
> ...
>
> set response header("Content-Disposition"; "attachment; filename=" +  
> $docFileName)
>
> delete response header("Pragma")
> delete response header("Cache-Control")
>
> set content type("application/octet-stream")
>
> write blob($blob_file; "application/octet-stream")
>
> ...
>
> This doesn't work ...
>
> I read or tried to read the RFC 2616, section 14.9, but ...
>
> Are the response header names wrong ?
>
> This should work ?
>
>
> Thanks again for your help!
>
>
> Regards,
>
>
> Raimon Fernandez
> Barcelona, Spain
>
>
> _______________________________________________
> Active4D-dev mailing list
> Active4D-dev@...
> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/
>

On 16/09/2009, at 16:33, Mike Vogt wrote:

> It turns out that IE can't correctly access Office documents from a  
> secure
> site!  This problem does not exist in Firefox, Opera, or Safari.  
> The work
> around is that the cache must be enabled for that request.  Adding  
> set cache
> control("Cache-Control: max-age=120, must-revalidate") just before  
> the set
> content type("application/vnd.ms-excel") fixed the problem.  The  
> following
> Microsoft knowledgebase article  documents the problem and the  
> solution.
>
> http://support.microsoft.com/kb/316431
>
>
> Thanks,
> Mike
>
>> -----Original Message-----
>> From: active4d-dev-bounces@...
>> [mailto:active4d-dev-bounces@...] On Behalf Of
>> Michael Bond
>> Sent: Wednesday, September 16, 2009 9:43 AM
>> To: active4d-dev@...
>> Subject: Re: [Active4d-dev] HTTPS in HTTP out
>>
>> 2009/9/16 Mike Vogt <mv@...>
>>
>>> Hi all,
>>> This might not be an Active4D question, but I'll ask
>> anyway.  I have a
>>> fuseaction that set content type("application/vnd.ms-excel") that
>>> worked great, asking the browser user if they wanted to
>> open in excel, etc.
>>> However, last night, we mad the entire site secure, so all urls are
>>> forced to https if they are http, using Apache rewrite in
>> from of 4D
>>> Server (we've always used Apache rewrite, just not for
>> https).  Now,
>>> all the excel links give an error saying 'Could not open
>> 'https://www.willsub.com/index...
>>> My question is, is there a response header or other info I
>> can set so
>>> that the response is set back to http from https?  Is there
>> any other
>>> way to look at this problem?
>>>
>>> TIA,
>>> Sincerely,
>>> Mike Vogt
>>> Willsub Technical Support
>>>
>>> I'll be honest I don't really understand the problem but if
>> you need a
>> secure request to be "transformed" under a certain rule set
>> to a standard non-secure then you could simply use a Redirect
>> from within Apache or Active4D to the non-secure pages with
>> the original request parameters attached to the Redirect.
>>
>> Michael Bond
>>
>> Michael
>> _______________________________________________
>> Active4D-dev mailing list
>> Active4D-dev@...
>> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
>> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/
>
> _______________________________________________
> Active4D-dev mailing list
> Active4D-dev@...
> http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
> Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/
>

_______________________________________________
Active4D-dev mailing list
Active4D-dev@...
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/