globalize_translations table

[Sprachprofi]

Hello,

first of all I'd like to thank you all very much for your work on the
Globalize Rails plugin. I found it very intuitive to use for my new
project, a multilingual social networking site. Way to go!

There are a couple issues I would like to address though, hoping
somebody has the answers.

1. In the sample translation interface (thanks for providing this!),
the translator always has to translate into whatever locale his
browser told the site to use. That's great for a default, but the
translator should be allowed to choose his target language. I have
spent a couple hours now trying to get a simple language picker to
work and there are two major obstacles: one is that there is no list
of locale codes, yet I can only change the locale knowing those codes.
Globalize_languages stores a whole lot of codes that probably won't be
used anywhere, but not even the most common locale codes! This is
annoying; I don't want to hard-code my list. The other obstacle is
that whenever I pick a locale in my controller, something resets it to
whatever the browser detects. The browser detect should only be done
when the user first enters the site and after that people should be
allowed to change their locale based on their preference.

2. While the pre-translated month names are really useful, the
.localize function still expects ME to provide the date format (such
as mm/dd/yyyy for American and dd.mm.yyyy for German and yyyy年mm月dd号
for Chinese). How am I supposed to know what the most common way of
writing a date is in whatever language, and why would I want to
hard-code that when .localize is supposed to take care of localizing
the date? I suggest the creation of functions like .localize_shortdate
that will output a correctly-formatted short date for the chosen
locale. Not as much variety as playing around with those variables
yourself, but at least you'll know that all your international users
will be able to understand the date. I can help with that if an
experienced globalize_rails plugin developer will guide me.

3. Is anybody using globalize_rails on a huge site? I believe that the
structure of globalize_translations is enough to make servers crack.
Why is the English text and the settings repeated for every single
translation? Is there no way of having a numerical index that would be
faster to search? Ideally, when a site gets really huge, each
language's texts would be completely separated into different tables,
so that they could be stored on different servers to reduce the load.

Just my two cents.

Best wishes,

Judith

[Sprachprofi]

Hello,

I still haven't got an answer, even though I wrote to this list
several weeks ago. I want to let you know that I'm really worried
about scalability - I'm programming a really big social network and
I'm very worried about how this will scale, seeing that texts from the
views in the database are identified by the English texts rather than
an id or something more searchable. If nobody can reassure me or give
me an example of a really popular website that successfully uses
Globalize, I will do a major fork / re-write for my company. We just
can't risk having to re-code this when translations refuse to scale to
keep up with our growth of users.

Best regards,

Judith Meyer

[Dan Coutu]

Judith, perhaps one thing that will help you out is to know that in fact
the globalize translations table can use anything you like for a key.
While all of the examples are complete English phrases it does not have
to be used that way. You can readily use a shorthand encoding mechanism
for each message and that will index (and therefore be looked up) much
faster than long phrases.

Note: if you do this then you will have to add "translations" for the
English text in addition to other languages. Then the "translated"
English will show up in place of the "keyword" English text.

Dan

Judith Meyer wrote:

> Hello,
>
> I still haven't got an answer, even though I wrote to this list
> several weeks ago. I want to let you know that I'm really worried
> about scalability - I'm programming a really big social network and
> I'm very worried about how this will scale, seeing that texts from the
> views in the database are identified by the English texts rather than
> an id or something more searchable. If nobody can reassure me or give
> me an example of a really popular website that successfully uses
> Globalize, I will do a major fork / re-write for my company. We just
> can't risk having to re-code this when translations refuse to scale to
> keep up with our growth of users.
>
> Best regards,
>
> Judith Meyer
>
>
>  

[virginian]

Hi Judith-

The following uses Swahili as a base_language US English as the default language and will give you a language picker using the native languages which gives it a cool look as the language fonts including CJK.

Let me know if you get stuck.

Best Regards-

Charles

in config/initializers/globalize.rb

include Globalize

Locale.set_base_language 'swa-SW' # Swahili is used to avoid messing up translations if we change a word in english

LOCALES = {

  'en'  => 'en-US',

  'es' => 'es-MX',

 'zh' => 'zh-CH'

}.freeze 

 

DEFAULT_LOCALE = 'en-US'

DEFAULT_DATE_FORMAT = '%H:%M:%S - %A %B %d %Y' 

# let rails handle international characters

$KCODE = 'u' 

require 'jcode'

#########

in your application.rb

 protected   

def set_locale

 # params overrides session, but en-US is only used if both

 # params and session are absent

 session[:locale] = LOCALES[params[:locale]] ||

                    session[:locale] ||

                    'en-US'

 Locale.set session[:locale] 

 

    rescue 

      Locale.set DEFAULT_LOCALE  

  end

  

  def set_charset 

    headers["Content-Type"] = "text/html; charset=utf-8" if headers["Content-Type"].blank? 

  end 

####

somewhere in your layout views

          <%= set_user_language  %>

#########

and in your application_helper.rb

  # Set the user language using the LOCALES from globalize.rb in config 

  # call by <%= set_user_language %>  

  #FIXME need to set locale.language_code for  the test envirement

  # added |code| link_to code.t, so that we can show Chinese character at the language bar. translate 'zh' at admin/translate)

  def set_user_language

    LOCALES.keys.map {|code| link_to code.t, url_for(:locale => code) }.join(" | ")  

  end

######

  

On Thu, May 15, 2008 at 1:32 PM, Dan Coutu <coutu@...> wrote:

Judith, perhaps one thing that will help you out is to know that in fact the globalize translations table can use anything you like for a key. While all of the examples are complete English phrases it does not have to be used that way. You can readily use a shorthand encoding mechanism for each message and that will index (and therefore be looked up) much faster than long phrases.

Note: if you do this then you will have to add "translations" for the English text in addition to other languages. Then the "translated" English will show up in place of the "keyword" English text.

Dan

Judith Meyer wrote:

Hello,

I still haven't got an answer, even though I wrote to this list
several weeks ago. I want to let you know that I'm really worried
about scalability - I'm programming a really big social network and
I'm very worried about how this will scale, seeing that texts from the
views in the database are identified by the English texts rather than
an id or something more searchable. If nobody can reassure me or give
me an example of a really popular website that successfully uses
Globalize, I will do a major fork / re-write for my company. We just
can't risk having to re-code this when translations refuse to scale to
keep up with our growth of users.

Best regards,

Judith Meyer


 

[Chris Cruft]

That is exactly what I do.
On 15 May 2008, at 13:32, Dan Coutu wrote:

> Judith, perhaps one thing that will help you out is to know that in  
> fact the globalize translations table can use anything you like for  
> a key. While all of the examples are complete English phrases it  
> does not have to be used that way. You can readily use a shorthand  
> encoding mechanism for each message and that will index (and  
> therefore be looked up) much faster than long phrases.
>
> Note: if you do this then you will have to add "translations" for  
> the English text in addition to other languages. Then the  
> "translated" English will show up in place of the "keyword" English  
> text.
>
> Dan
>
> Judith Meyer wrote:
>> Hello,
>>
>> I still haven't got an answer, even though I wrote to this list
>> several weeks ago. I want to let you know that I'm really worried
>> about scalability - I'm programming a really big social network and
>> I'm very worried about how this will scale, seeing that texts from  
>> the
>> views in the database are identified by the English texts rather than
>> an id or something more searchable. If nobody can reassure me or give
>> me an example of a really popular website that successfully uses
>> Globalize, I will do a major fork / re-write for my company. We just
>> can't risk having to re-code this when translations refuse to scale  
>> to
>> keep up with our growth of users.
>>
>> Best regards,
>>
>> Judith Meyer
>>
>>
>>
>

[Sprachprofi]

Thank you very much for all your responses!

I guess the problem is that I really like the implicit working of
Globalize, that the other programmers can just type "You have been
successfully registered.".t and that string will be automatically
looked up and/or added to the database for translation. Using a
different tr_key there, ideally an integer id for database purposes,
would make the code much less readable and the addition of new strings
or changing of old ones would become a pain, unless there's something
I'm not aware of.
So I thought of a solution that is both readable and scalable: in
addition to allowing strings like "You have been successfully
registered".t , I'd also allow "#115#> You have been successfully
registered".t, where 115 is the tr_key extracted by Globalize. Then
people or a script can periodically go over strings and add in those
ids to speed up retrieval from the database (code-writing code would
be better - LISP anyone?). There are three imaginable possibilities of
handling strings that don't have an id yet:
1. searching for the original text in the "text" column to find the
id, and using that id to find the translation (2 database calls)
2. storing the original text in a new "orig_text" column that is not
accessed otherwise, so that translations can be retrieved using one
call only
3. ignoring them and returning the original text

I'm not sure which is the best. Anyway, what do you you think of this idea?

Also, there was another suggestion for improvement in my original e-mail:
<quote>
While the pre-translated month names are really useful, the .localize
function still expects ME to provide the date format (such as
mm/dd/yyyy for American and dd.mm.yyyy for German and yyyy年mm月dd号 for
Chinese). How am I supposed to know what the most common way of
writing a date is in whatever language, and why would I want to
hard-code that when .localize is supposed to take care of localizing
the date? I suggest the creation of functions like .localize_shortdate
that will output a correctly-formatted short date for the chosen
locale. Not as much variety as playing around with those variables
yourself, but at least you'll know that all your international users
will be able to understand the date. I can help with that if an
experienced globalize_rails plugin developer will guide me.
</quote>

Best wishes,

Judith

[Jérôme Lipowicz]

2008/5/16 Judith Meyer <yutian.mei@...>:

> Thank you very much for all your responses!
>
> I guess the problem is that I really like the implicit working of
> Globalize, that the other programmers can just type "You have been
> successfully registered.".t and that string will be automatically
> looked up and/or added to the database for translation. Using a
> different tr_key there, ideally an integer id for database purposes,
> would make the code much less readable and the addition of new strings
> or changing of old ones would become a pain, unless there's something
> I'm not aware of.

You could also set default strings such as "successful_registration".t
then refactore them in english even if english is the base language.

> So I thought of a solution that is both readable and scalable: in
> addition to allowing strings like "You have been successfully
> registered".t , I'd also allow "#115#> You have been successfully
> registered".t, where 115 is the tr_key extracted by Globalize. Then
> people or a script can periodically go over strings and add in those
> ids to speed up retrieval from the database (code-writing code would
> be better - LISP anyone?). There are three imaginable possibilities of
> handling strings that don't have an id yet:
> 1. searching for the original text in the "text" column to find the
> id, and using that id to find the translation (2 database calls)
> 2. storing the original text in a new "orig_text" column that is not
> accessed otherwise, so that translations can be retrieved using one
> call only
> 3. ignoring them and returning the original text
>
> I'm not sure which is the best. Anyway, what do you you think of this idea?
>

Ids as integers are fine for a quick database access, but from a
semantic point of view, I would never tolerate in my code #115# : this
is just unreadable and unmaintanable.

I would prefer setting up symbols like :successfull_registration.t


--
Jérôme Lipowicz

[Chris Cruft]

Globalize's feature of translating strings may not be appropriate for  
a whole page with numerous strings -just too much DB overhead.  
Fortunately, there is a mechanism to pre-translate the page using  
locale-specific templates.  I would consider that if performance is a  
concern.

-Chris

On 16 May 2008, at 5:14, Judith Meyer wrote:

> Thank you very much for all your responses!
>
> I guess the problem is that I really like the implicit working of
> Globalize, that the other programmers can just type "You have been
> successfully registered.".t and that string will be automatically
> looked up and/or added to the database for translation. Using a
> different tr_key there, ideally an integer id for database purposes,
> would make the code much less readable and the addition of new strings
> or changing of old ones would become a pain, unless there's something
> I'm not aware of.
> So I thought of a solution that is both readable and scalable: in
> addition to allowing strings like "You have been successfully
> registered".t , I'd also allow "#115#> You have been successfully
> registered".t, where 115 is the tr_key extracted by Globalize. Then
> people or a script can periodically go over strings and add in those
> ids to speed up retrieval from the database (code-writing code would
> be better - LISP anyone?). There are three imaginable possibilities of
> handling strings that don't have an id yet:
> 1. searching for the original text in the "text" column to find the
> id, and using that id to find the translation (2 database calls)
> 2. storing the original text in a new "orig_text" column that is not
> accessed otherwise, so that translations can be retrieved using one
> call only
> 3. ignoring them and returning the original text
>
> I'm not sure which is the best. Anyway, what do you you think of  
> this idea?
>
> Also, there was another suggestion for improvement in my original e-
> mail:
> <quote>
> While the pre-translated month names are really useful, the .localize
> function still expects ME to provide the date format (such as
> mm/dd/yyyy for American and dd.mm.yyyy for German and yyyy年mm
> 月dd号 for
> Chinese). How am I supposed to know what the most common way of
> writing a date is in whatever language, and why would I want to
> hard-code that when .localize is supposed to take care of localizing
> the date? I suggest the creation of functions like .localize_shortdate
> that will output a correctly-formatted short date for the chosen
> locale. Not as much variety as playing around with those variables
> yourself, but at least you'll know that all your international users
> will be able to understand the date. I can help with that if an
> experienced globalize_rails plugin developer will guide me.
> </quote>
>
> Best wishes,
>
> Judith
>

[Sprachprofi]

Hi again,

> You could also set default strings such as "successful_registration".t
> then refactore them in english even if english is the base language.

I want to avoid unnecessary database calls; English strings don't need
to be translated to English.

>> So I thought of a solution that is both readable and scalable: in
>> addition to allowing strings like "You have been successfully
>> registered".t , I'd also allow "#115#> You have been successfully
>> registered".t, where 115 is the tr_key extracted by Globalize. Then
>> people or a script can periodically go over strings and add in those
>> ids to speed up retrieval from the database (code-writing code would
>> be better - LISP anyone?). There are three imaginable possibilities of
>> handling strings that don't have an id yet:
>> 1. searching for the original text in the "text" column to find the
>> id, and using that id to find the translation (2 database calls)
>> 2. storing the original text in a new "orig_text" column that is not
>> accessed otherwise, so that translations can be retrieved using one
>> call only
>> 3. ignoring them and returning the original text
>>
>> I'm not sure which is the best. Anyway, what do you you think of this idea?
>>
>

> Ids as integers are fine for a quick database access, but from a
> semantic point of view, I would never tolerate in my code #115# : this
> is just unreadable and unmaintanable.
>
> I would prefer setting up symbols like :successfull_registration.t

I believe there may be a misunderstanding here: in the code I still
want to see the entire message that the user will see. That's why I
came up with coupled strings that consist of an id plus the entire
message - the message is there for ease of reading and the id is there
for faster database calls.

Je pense que peut-être je n'ai pas été assez claire: dans le code je
voudrais bien voir le message entier, par example "You have
successfully registered".t, pas seulement :successfull_registration.t
(aussi parce que c'est difficile à éditer après, il faut aller dans la
base de données et c'est ca que je veux éviter) et certainement pas
seulement "#115#".t !  Alors je crée des strings qui contiennent l'id,
mais qui contiennent aussi le message entier, comme "#115#> You have
been successfully registered".t - ca serait un string comme ca, et
Globalize ferait le découpage. De telle facon, on pourrait toujours
lire le message et le modifier, mais Globalize ne ferait pas la
recherche de tr_key = "You have been successfully registered" dans la
base de données, Globalize chercherait seulement pour les traductions
de tr_key = 115. Et si le wording change, les traductions sont
toujours trouvables par tr_key 115.

Amicalement,

Judith

[ginger72]

It sounds as if you may want to reconsider your choice for globalize,  
maybe gettext is the tool of choice for your requirements?  
Unfortunately, there is no one i18n solution fits all in Rails today.  
Internationalization support like that in Java would be great.

I am using Globalize and gettext in different projects, each has its  
strength and weakness.

Tschüß,
Jürgen

On May 16, 2008, at 8:48 AM, Judith Meyer wrote:

> Hi again,
>
>> You could also set default strings such as  
>> "successful_registration".t
>> then refactore them in english even if english is the base language.
>
> I want to avoid unnecessary database calls; English strings don't need
> to be translated to English.
>
>>> So I thought of a solution that is both readable and scalable: in
>>> addition to allowing strings like "You have been successfully
>>> registered".t , I'd also allow "#115#> You have been successfully
>>> registered".t, where 115 is the tr_key extracted by Globalize. Then
>>> people or a script can periodically go over strings and add in those
>>> ids to speed up retrieval from the database (code-writing code would
>>> be better - LISP anyone?). There are three imaginable  
>>> possibilities of
>>> handling strings that don't have an id yet:
>>> 1. searching for the original text in the "text" column to find the
>>> id, and using that id to find the translation (2 database calls)
>>> 2. storing the original text in a new "orig_text" column that is not
>>> accessed otherwise, so that translations can be retrieved using one
>>> call only
>>> 3. ignoring them and returning the original text
>>>
>>> I'm not sure which is the best. Anyway, what do you you think of  
>>> this idea?
>>>
>>
>
>> Ids as integers are fine for a quick database access, but from a
>> semantic point of view, I would never tolerate in my code #115# :  
>> this
>> is just unreadable and unmaintanable.
>>
>> I would prefer setting up symbols like :successfull_registration.t
>
> I believe there may be a misunderstanding here: in the code I still
> want to see the entire message that the user will see. That's why I
> came up with coupled strings that consist of an id plus the entire
> message - the message is there for ease of reading and the id is there
> for faster database calls.
>
> Je pense que peut-être je n'ai pas été assez claire: dans le code je
> voudrais bien voir le message entier, par example "You have
> successfully registered".t, pas seulement :successfull_registration.t
> (aussi parce que c'est difficile à éditer après, il faut aller dans la
> base de données et c'est ca que je veux éviter) et certainement pas
> seulement "#115#".t !  Alors je crée des strings qui contiennent l'id,
> mais qui contiennent aussi le message entier, comme "#115#> You have
> been successfully registered".t - ca serait un string comme ca, et
> Globalize ferait le découpage. De telle facon, on pourrait toujours
> lire le message et le modifier, mais Globalize ne ferait pas la
> recherche de tr_key = "You have been successfully registered" dans la
> base de données, Globalize chercherait seulement pour les traductions
> de tr_key = 115. Et si le wording change, les traductions sont
> toujours trouvables par tr_key 115.
>
> Amicalement,
>
> Judith

[Julio Cesar C Neto-2]

Chris, which mechanism are you talking about? How can I use this pre-translate stuff?

Thanks
Julio Cesar

2008/5/16 Chris Hapgood <cch1@...>:

Globalize's feature of translating strings may not be appropriate for a whole page with numerous strings -just too much DB overhead.  Fortunately, there is a mechanism to pre-translate the page using locale-specific templates.  I would consider that if performance is a concern.

-Chris

On 16 May 2008, at 5:14, Judith Meyer wrote:

Thank you very much for all your responses!

I guess the problem is that I really like the implicit working of
Globalize, that the other programmers can just type "You have been
successfully registered.".t and that string will be automatically
looked up and/or added to the database for translation. Using a
different tr_key there, ideally an integer id for database purposes,
would make the code much less readable and the addition of new strings
or changing of old ones would become a pain, unless there's something
I'm not aware of.
So I thought of a solution that is both readable and scalable: in
addition to allowing strings like "You have been successfully
registered".t , I'd also allow "#115#> You have been successfully
registered".t, where 115 is the tr_key extracted by Globalize. Then
people or a script can periodically go over strings and add in those
ids to speed up retrieval from the database (code-writing code would
be better - LISP anyone?). There are three imaginable possibilities of
handling strings that don't have an id yet:
1. searching for the original text in the "text" column to find the
id, and using that id to find the translation (2 database calls)
2. storing the original text in a new "orig_text" column that is not
accessed otherwise, so that translations can be retrieved using one
call only
3. ignoring them and returning the original text

I'm not sure which is the best. Anyway, what do you you think of this idea?

Also, there was another suggestion for improvement in my original e-mail:
<quote>
While the pre-translated month names are really useful, the .localize
function still expects ME to provide the date format (such as
mm/dd/yyyy for American and dd.mm.yyyy for German and yyyy年mm月dd号 for
Chinese). How am I supposed to know what the most common way of
writing a date is in whatever language, and why would I want to
hard-code that when .localize is supposed to take care of localizing
the date? I suggest the creation of functions like .localize_shortdate
that will output a correctly-formatted short date for the chosen
locale. Not as much variety as playing around with those variables
yourself, but at least you'll know that all your international users
will be able to understand the date. I can help with that if an
experienced globalize_rails plugin developer will guide me.
</quote>

Best wishes,

Judith

[Chris Cruft]

Julio,

I'm virtually certain that you can create a template with a filename like this:

show.fr.rhtml

and it will be rendered instead of the standard show.rhtml if your locale if 'fr'

I have not personally used this functionality in a long time, but it should work on Rails 2.0x and 1.2x.

Caveat Programmer: This is guaranteed to NOT work on Edge -there have been many changes dealing with template handling that will appear in Rails 2.1 and currently Globalize does not support them.

Regards,

Chris

On 16 May 2008, at 15:44, Julio Cesar C Neto wrote:

Chris, which mechanism are you talking about? How can I use this pre-translate stuff?

Thanks
Julio Cesar

2008/5/16 Chris Hapgood <cch1@...>:

Globalize's feature of translating strings may not be appropriate for a whole page with numerous strings -just too much DB overhead.  Fortunately, there is a mechanism to pre-translate the page using locale-specific templates.  I would consider that if performance is a concern.

-Chris

On 16 May 2008, at 5:14, Judith Meyer wrote:

Thank you very much for all your responses!

I guess the problem is that I really like the implicit working of
Globalize, that the other programmers can just type "You have been
successfully registered.".t and that string will be automatically
looked up and/or added to the database for translation. Using a
different tr_key there, ideally an integer id for database purposes,
would make the code much less readable and the addition of new strings
or changing of old ones would become a pain, unless there's something
I'm not aware of.
So I thought of a solution that is both readable and scalable: in
addition to allowing strings like "You have been successfully
registered".t , I'd also allow "#115#> You have been successfully
registered".t, where 115 is the tr_key extracted by Globalize. Then
people or a script can periodically go over strings and add in those
ids to speed up retrieval from the database (code-writing code would
be better - LISP anyone?). There are three imaginable possibilities of
handling strings that don't have an id yet:
1. searching for the original text in the "text" column to find the
id, and using that id to find the translation (2 database calls)
2. storing the original text in a new "orig_text" column that is not
accessed otherwise, so that translations can be retrieved using one
call only
3. ignoring them and returning the original text

I'm not sure which is the best. Anyway, what do you you think of this idea?

Also, there was another suggestion for improvement in my original e-mail:
<quote>
While the pre-translated month names are really useful, the .localize
function still expects ME to provide the date format (such as
mm/dd/yyyy for American and dd.mm.yyyy for German and yyyy年mm月dd号 for
Chinese). How am I supposed to know what the most common way of
writing a date is in whatever language, and why would I want to
hard-code that when .localize is supposed to take care of localizing
the date? I suggest the creation of functions like .localize_shortdate
that will output a correctly-formatted short date for the chosen
locale. Not as much variety as playing around with those variables
yourself, but at least you'll know that all your international users
will be able to understand the date. I can help with that if an
experienced globalize_rails plugin developer will guide me.
</quote>

Best wishes,

Judith