UTF 8 encoding

[nitesh rijal]

Hello

I am using Koha 3.0.3 in Debian Lenny Stable. I am doing fine with everything else, the problem that I'm facing is that whenever I import some UTF 8 encoded data into the database, it doesnot show the decoded text in the search result. It keeps on showing the encoded data.

I have selected collation as utf8_general_ci and the actual data is in Nepali Unicode ( Devnagari). What is wrong here, I can't figure it out.

Please Help.

Thankyou.

--
Nitesh Rijal
BE IT
http://niteshrijal.com.np
[phone]

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Chris Nighswonger]

Hi Nitesh,

Checkout http://wiki.koha.org/doku.php?id=encodingscratchpad as there
might be some helpful info there.

Kind Regards,
Chris

2009/11/3 nitesh rijal <rijal.it@...>:

> Hello
>
> I am using Koha 3.0.3 in Debian Lenny Stable. I am doing fine with
> everything else, the problem that I'm facing is that whenever I import some
> UTF 8 encoded data into the database, it doesnot show the decoded text in
> the search result. It keeps on showing the encoded data.
>
> I have selected collation as utf8_general_ci and the actual data is in
> Nepali Unicode ( Devnagari). What is wrong here, I can't figure it out.
>
> Please Help.
>
> Thankyou.
>
> --
> Nitesh Rijal
> BE IT
> http://niteshrijal.com.np
> [phone]
>
> _______________________________________________
> Koha mailing list
> Koha@...
> http://lists.katipo.co.nz/mailman/listinfo/koha
>
>

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Beda Szukics-2]

Chris Nighswonger schrieb:
> Checkout http://wiki.koha.org/doku.php?id=encodingscratchpad as there
> might be some helpful info there.

I noticed that this hint is missing in INSTALL.debian-lenny. In
INSTALL.debian it's there. Any reason for this difference?

Greetings         Beda
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Chris Nighswonger]

Hi Beda,

On Tue, Nov 3, 2009 at 8:20 AM, Beda Szukics <beda@...> wrote:
> Chris Nighswonger schrieb:
>> Checkout http://wiki.koha.org/doku.php?id=encodingscratchpad as there
>> might be some helpful info there.
>
> I noticed that this hint is missing in INSTALL.debian-lenny. In
> INSTALL.debian it's there. Any reason for this difference?
>

Probably just an oversight.

Kind Regards,
Chris
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[nitesh rijal]

 actually this is what  i got when i used UTF8_decode function of PHP

CODE:
<?php
    $conn = mysql_connect('localhost','root','healthnet') or die('Could not connect to database');
    $db = mysql_select_db('koha',$conn) or die('Could not select database');
    $query = mysql_query("SELECT author FROM biblio") or die('Could not execute query');
    $row = mysql_fetch_array($query);
    echo '<pre>'.$row['author'].'</pre>';
    echo '<pre>'.utf8_decode($row['author']).'</pre>';
?>
   
Result:

"कारॠकी, यॠवराज सिंह"

"�?ार्�?�?, युवरा�? सि�?ह"

--
Nitesh Rijal
BE IT
http://niteshrijal.com.np
[phone]

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[nitesh rijal]

actually this is what  i got when i used UTF8_decode function of PHP 

CODE:
<?php
    $conn = mysql_connect('localhost','root','healthnet') or die('Could not connect to database');
    $db = mysql_select_db('koha',$conn) or die('Could not select database');
    $query = mysql_query("SELECT author FROM biblio") or die('Could not execute query');
    $row = mysql_fetch_array($query);
    echo '<pre>'.$row['author'].'</pre>';
    echo '<pre>'.utf8_decode($row['author']).'</pre>';
?>
    
Result:

"कारॠकी, यॠवराज सिंह"

"�?ार्�?�?, युवरा�? सि�?ह"

Is there a problem with the search.pl script file???

I have set up the mysql character set as UTF8 and also the collation in phpmyadmin is utf8_general_ci . Isn't this sufficient for the Nepali Unicode???

Nepali is a Devnagari Script, similar to Hindi.

Please Help.

Thanks.

--
Nitesh Rijal
BE IT
http://niteshrijal.com.np
[phone]

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Chris Nighswonger]

Hi Nitesh,

On Wed, Nov 4, 2009 at 6:02 AM, nitesh rijal <rijal.it@...> wrote:

> actually this is what  i got when i used UTF8_decode function of PHP
>
> CODE:
> <?php
>     $conn = mysql_connect('localhost','root','healthnet') or die('Could not
> connect to database');
>     $db = mysql_select_db('koha',$conn) or die('Could not select database');
>     $query = mysql_query("SELECT author FROM biblio") or die('Could not
> execute query');
>     $row = mysql_fetch_array($query);
>     echo '<pre>'.$row['author'].'</pre>';
>     echo '<pre>'.utf8_decode($row['author']).'</pre>';
> ?>
>
> Result:
>
> "कारॠकी, यॠवराज सिंह"
>
> "�?ार्�?�?, युवरा�? सि�?ह"
>
> Is there a problem with the search.pl script file???
> I have set up the mysql character set as UTF8 and also the collation in
> phpmyadmin is utf8_general_ci . Isn't this sufficient for the Nepali
> Unicode???
> Nepali is a Devnagari Script, similar to Hindi.

I am not familiar with PHP.

UTF8 setup is a bit tricky. As the wiki link points out, every
component must be set to support UTF8. The OS, MySQL server, MySQL
client, Apache, etc.

There are many libraries in India using Koha successfully. Perhaps one
of them could help you with getting things setup correctly.

Kind Regards,
Chris
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[nitesh rijal]

I have setup everything as the WIKI points. I got all the results
similar as well. My only problem is that the data stored in the
database is in encoded form and I need it displayed in decoded form
while searching the string.

The string in database looks like this:
"कारॠकी, यॠवराज सिंह"

The string I got from using an external utf8_decode() in a different PHP script.
"�?ार्�?�?, युवरा�? सि�?ह"

This is what I did.

CODE:
<?php
    $conn = mysql_connect('localhost','root','healthnet') or
die('Could not connect to database');
    $db = mysql_select_db('koha',$conn) or die('Could not select database');
    $query = mysql_query("SELECT author FROM biblio") or die('Could
not execute query');
    $row = mysql_fetch_array($query);
    echo '<pre>'.$row['author'].'</pre>';
    echo '<pre>'.utf8_decode($row['author']).'</pre>';
?>

Is there a problem with the search.pl script file???
I have set up the mysql character set as UTF8 and also the collation
in phpmyadmin is utf8_general_ci . Isn't this sufficient for the
Nepali Unicode???
Nepali is a Devnagari Script, similar to Hindi.


Please Help.


Thanks.

On 11/4/09, Chris Nighswonger <cnighswonger@...> wrote:

> Hi Nitesh,
>
> On Wed, Nov 4, 2009 at 6:02 AM, nitesh rijal <rijal.it@...> wrote:
>> actually this is what  i got when i used UTF8_decode function of PHP
>>
>> CODE:
>> <?php
>>     $conn = mysql_connect('localhost','root','healthnet') or die('Could
>> not
>> connect to database');
>>     $db = mysql_select_db('koha',$conn) or die('Could not select
>> database');
>>     $query = mysql_query("SELECT author FROM biblio") or die('Could not
>> execute query');
>>     $row = mysql_fetch_array($query);
>>     echo '<pre>'.$row['author'].'</pre>';
>>     echo '<pre>'.utf8_decode($row['author']).'</pre>';
>> ?>
>>
>> Result:
>>
>> "कारॠकी, यॠवराज सिंह"
>>
>> "�?ार्�?�?, युवरा�? सि�?ह"
>>
>> Is there a problem with the search.pl script file???
>> I have set up the mysql character set as UTF8 and also the collation in
>> phpmyadmin is utf8_general_ci . Isn't this sufficient for the Nepali
>> Unicode???
>> Nepali is a Devnagari Script, similar to Hindi.
>
> I am not familiar with PHP.
>
> UTF8 setup is a bit tricky. As the wiki link points out, every
> component must be set to support UTF8. The OS, MySQL server, MySQL
> client, Apache, etc.
>
> There are many libraries in India using Koha successfully. Perhaps one
> of them could help you with getting things setup correctly.
>
> Kind Regards,
> Chris
>

--
Nitesh Rijal
BE IT
http://niteshrijal.com.np
[phone]
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha