[Tickets #8685] Json utf8 respons object

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Created By         | roman@...
  Summary            | Json utf8 respons object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Unconfirmed
  Priority           | 2. Medium
  Milestone          | Horde 3.3.5
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-06 12:35) wrote:

In Horde 3.3.5 the response object from Kronolith is using wrong charset.

If Horde API is used to Power External Sites like this one:
http://theupstairsroom.com/67

Problem:
Strings get cut off

Reason:
kronolith/lib/Driver/sql.php (line 453)
Kronolith_Event_sql turns all utf8 strings into  
$GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1';


Solution:

/lib/Horde/RPC/jsonrpc.php
class Horde_RPC_jsonrpc();
function getResponse($request);

Line 91
--   return Horde_Serialize::serialize($response, SERIALIZE_JSON);
++ return Horde_Serialize::serialize($response, SERIALIZE_JSON,  
$GLOBALS['_HORDE_STRING_CHARSET']);






--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | Michael Rubinsky <mrubinsk@...>
  Summary            | Json utf8 respons object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
-State              | Unconfirmed
+State              | Feedback
  Priority           | 2. Medium
  Milestone          | Horde 3.3.5
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Michael Rubinsky <mrubinsk@...> (2009-11-06 14:09) wrote:

> In Horde 3.3.5 the response object from Kronolith is using wrong charset.
>
> If Horde API is used to Power External Sites like this one:
> http://theupstairsroom.com/67
>
> Problem:
> Strings get cut off
>
> Reason:
> kronolith/lib/Driver/sql.php (line 453)
> Kronolith_Event_sql turns all utf8 strings into  
> $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1';

What version of Kronolith is this? That line number has nothing to do  
with charset conversion, and furthermore, nowhere do we use  
$GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local  
modification?






--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 respons object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | Horde 3.3.5
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-07 07:46) wrote:

>> In Horde 3.3.5 the response object from Kronolith is using wrong charset.
>>
>> If Horde API is used to Power External Sites like this one:
>> http://theupstairsroom.com/67
>>
>> Problem:
>> Strings get cut off
>>
>> Reason:
>> kronolith/lib/Driver/sql.php (line 453)
>> Kronolith_Event_sql turns all utf8 strings into
>> $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1';
>
> What version of Kronolith is this?

Horde Groupware 1.2.4
Kronolith H3 (2.3.2)
kronolith/lib/api.php,v 1.126.2.64 2009/09/04 10:38:36 jan Exp $

> That line number has nothing to do with charset conversion,
> and furthermore, nowhere do we use  
> $GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local  
> modification?

Procedure during the a json call server side:

1. json is calling function &_kronolith_eventFromUID($uid) -->  
kronolith/lib/api.php line 1133
2. the event object get fetched:  $event =  
$GLOBALS['kronolith_driver']->getByUID($uid) --> kronolith/lib/api.php  
line 1137
-------------
    Get an event or events with the given UID value in  
kronolith/lib/Driver/sql.php,v 1.213 2008/10/18 04:20:42
    3. call function: &getByUID($uid, $calendars = null, $getAll =  
false) --> kronolith/lib/Driver/sql.php line 416
    4. the event get cached :  
$this->_cache[$this->_calendar][$event['event_id']] = &new  
Kronolith_Event_sql($this, $event) --> kronolith/lib/Driver/sql.php  
line 453
    5. call fromDriver(SQLEvent) in the class Kronolith_Event_sql  
method fromDriver($SQLEvent) kronolith/lib/Driver/sql.php line 871
    6. call $driver->convertFromDriver($SQLEvent['event_description'])  
--> kronolith/lib/Driver/sql.php line 948
    7. Converts a value from the default charset to the driver's  
charset String::convertCharset($value, $this->_params['charset'])  
kronolith/lib/Driver/sql.php line 803
------------
       The String:: class provides static methods for charset and  
locale safe framework/Util/String.php,v 1.43.6.37 2009/03/30 15:31:38
       8.call function convertCharset($input, $from, $to = null)  
framework/Util/String.php line 76
       on top of this file the $GLOBALS['_HORDE_STRING_CHARSET'] =  
'iso-8859-1'; is set.
------------
This is way we get an iso-8859-1 formated event back from kronolith.

json does not like iso-8859-1.This leads us back on track:
------------
framework/RPC/RPC/jsonrpc.php,v 1.9 2009/02/21 01:58:14 chuck Exp

call function getResponse($request) --> framework/RPC/RPC/jsonrpc.php line 47
return Horde_Serialize::serialize($response, Horde_Serialize::JSON)  
--> framework/RPC/RPC/jsonrpc.php line 88

       We have to make sure, that all strings in this responseheader are utf8.
       for this we should add the propper charset in  
Horde_Serialize::serialize($response, Horde_Serialize::JSON,  
"CHARSET") <---
       or make somehow sure, that we get all respondehaeder for json utf8.

------------
    $Horde: framework/Serialize/Serialize.php,v 1.25.10.16 2009/01/06  
15:23:34 jan Exp $

    // $params = Source character set
    case SERIALIZE_JSON in function _serialize($data, $mode, $params =  
null) --> framework/Serialize/Serialize.php line 264


regards roman





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | Jan Schneider <jan@...>
-Summary            | Json utf8 respons object
+Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
-Milestone          | Horde 3.3.5
+Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Jan Schneider <jan@...> (2009-11-07 07:55) wrote:

Please try what I committed.





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-07 14:53) wrote:

> Changes have been made in CVS for this ticket:
>
> JSON is always UTF-8 (Bug #8685).
> http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.4&r2=1.4.2.5&ty=u

this breaks the json request.
--> Horde_NLS::setCharsetEnvironment('UTF-8');





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | Jan Schneider <jan@...>
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Jan Schneider <jan@...> (2009-11-08 05:38) wrote:

> this breaks the json request.

Care to tell us how?





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-08 05:44) wrote:

>> this breaks the json request.
>
> Care to tell us how?
under investigation.... right now...

I let you know as soon as i know more...






--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 05:14) wrote:

>>> this breaks the json request.
>>
>> Care to tell us how?
[09-Nov-2009 11:06:04] PHP Fatal error:  Class 'Horde_NLS' not found  
in /horde3/lib/Horde/RPC/jsonrpc.php on line 44






--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 05:43) wrote:

framework/RPC/RPC/jsonrpc.php,v 1.4.2.4 2009/01/06 15:23:32

Line 91
+ return Horde_Serialize::serialize($response, SERIALIZE_JSON,  
NLS::getCharset());
- return Horde_Serialize::serialize($response, SERIALIZE_JSON);

Works for me.





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | Jan Schneider <jan@...>
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Jan Schneider <jan@...> (2009-11-09 08:21) wrote:

How about now?

> Line 91
> + return Horde_Serialize::serialize($response, SERIALIZE_JSON,  
> NLS::getCharset());
> - return Horde_Serialize::serialize($response, SERIALIZE_JSON);

There is no such line.





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 08:40) wrote:

> How about now?
>
>> Line 91
>> + return Horde_Serialize::serialize($response, SERIALIZE_JSON,
>> NLS::getCharset());
>> - return Horde_Serialize::serialize($response, SERIALIZE_JSON);
>
> There is no such line.

Jsonrpc.php, which is delivered 2009-09-14: Horde Groupware 1.2.4:

Might be this one:
http://cvs.horde.org/h/chora/annotate.php/framework/RPC/RPC/Attic/jsonrpc.php?sa=1&rev=1.5









--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 08:59) wrote:

> Changes have been made in CVS for this ticket:
>
> Fix class name (Bug #8685).
> http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.5&r2=1.4.2.6&ty=u

This does not change the charsetcoding of strings in $response.
@return string  The JSON encoded response from the server.

Still the same behavior for me.





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | Jan Schneider <jan@...>
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Jan Schneider <jan@...> (2009-11-09 09:14) wrote:

Please download a fresh copy of this file, revision 1.4.2.6. You might  
still have local modifications that interfere.





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 09:33) wrote:

> Please download a fresh copy of this file, revision 1.4.2.6. You  
> might still have local modifications that interfere.
Still no luck. Strings get cut off.  $response is still in ISO-8859-1
Will do a fresh install.  Hope that helps.










--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | nd@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


nd@... (2009-11-09 11:13) wrote:

> In Horde 3.3.5 the response object from Kronolith is using wrong charset.
>
> If Horde API is used to Power External Sites like this one:
> http://theupstairsroom.com/67
>
> Problem:
> Strings get cut off
>
> Reason:
> kronolith/lib/Driver/sql.php (line 453)
> Kronolith_Event_sql turns all utf8 strings into  
> $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1';

I can confirm this solution (thanks to Roman too). It solves the  
described problem for me (had that problems with strings containing  
german 'umlauts') - i use JSON with python-simplejson (in ZOPE)  
externally. With XMLRPC i had not such charset problems.

good work ;)

hth
cheers,


Niels
---
Niels Dettenbach
http://www.syndicat.com
http://www.stadtradio-goettingen.de





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...

[bugs-14]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/8685
------------------------------------------------------------------------------
  Ticket             | 8685
  Updated By         | roman@...
  Summary            | Json utf8 response object
  Queue              | Horde Framework Packages
  Version            | FRAMEWORK_3
  Type               | Bug
  State              | Feedback
  Priority           | 2. Medium
  Milestone          | 3.3.6
  Patch              |
  Owners             |
------------------------------------------------------------------------------


roman@... (2009-11-09 12:12) wrote:

I can see clearly now what the patch of Jan is trying to do. :-)

The muche smarter way as i tried. Nomore back and forth conversions.
In theory this should fix the issue.

Unfortunately in the setCharsetEnvironment(){} function is another "bug".
(in NLS.php)

the NLS::setLang(); turns the charset back to iso,
before the utf-8 can take action.

Regards roman





--
You are subscribed to this list as: lists@...
To unsubscribe, mail: bugs-unsubscribe@...