improving our translations process

[philipo]

Greetings doc geeks,

We need to improve our translations system for various reasons:

A. It's difficult to work with:
Honestly I don't know how translators put up with the current system  
and think it helps contribute to dead translations. Many kudos go to  
the translators!

B. A new VCS is forcing a change:
One day soon the PHP project will move from CVS to a new VCS, likely  
either Subversion or Git. This presents a real problem because we rely  
on CVS revision numbers for tracking the translations status.

C. Translating markup and related worries:
Whether or not an XML markup change affects translations shouldn't be  
on our minds, but today it must be. Currently all changes, markup or  
otherwise, affect the status of a translation. Ideas were explored to  
help solve this symptom but nothing great came about.

So we must change our process, but how? We have a few options:

1. Keep the current system, and simply change the revision handling
2. The same as (1) but with a few other tweaks
3. Totally rethink the system

I'm in favor of (3) but personally don't know the topic well enough to  
comfortably design it. My gut says move towards a gettext based system  
with po/pot files but again it's not something I've used personally  
but do know others have and like it, that other large manuals use it  
with success, and that many tools exist to manage this including po  
specific text editors. And because several projects already do this we  
could evaluate several and decide what would work best for us. And, of  
course steal their HOWTO information ;)

Please, all comments, complaints, questions and suggestions are  
welcome. Especially from translators!

Regards,
Philip

P.S. Please only reply to phpdoc@..., this initial email CCs  
all doc lists to notify everyone of the topic.

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Benjamin-30]

Hi
       
Vote for Option 3.
We have to evolve, it is obsolete CVS, SVN (subversion) is much better, easy
editing and easy management of groups of translation.

The documentation format should be using docbook XML.

Benjamin Gonzales
http://codigolinea.com 
http://zfdes.com



-----Mensaje original-----
De: Philip Olson [mailto:philip@...]
Enviado el: Friday, April 10, 2009 3:59 PM
Para: PHP Documentation ML
CC: doc-bg@...; doc-da@...; doc-cs@...;
doc-fi@...; doc-fr@...; doc-el@...;
doc-he@...; doc-de@...; doc-hu@...;
doc-hk@...; doc-pt-br@...; doc-nl@...;
doc-zh@...; doc-tw@...; doc-ar@...;
doc-it@...; doc-ja@...; doc-kr@...;
doc-no@...; doc-fa@...; doc-pl@...;
doc-pt@...; doc-ro@...; doc-ru@...;
doc-se@...; doc-sk@...; doc-sl@...;
doc-es@...; doc-sv@...; doc-tr@...
Asunto: [DOC-ES] improving our translations process

Greetings doc geeks,

We need to improve our translations system for various reasons:

A. It's difficult to work with:
Honestly I don't know how translators put up with the current system  
and think it helps contribute to dead translations. Many kudos go to  
the translators!

B. A new VCS is forcing a change:
One day soon the PHP project will move from CVS to a new VCS, likely  
either Subversion or Git. This presents a real problem because we rely  
on CVS revision numbers for tracking the translations status.

C. Translating markup and related worries:
Whether or not an XML markup change affects translations shouldn't be  
on our minds, but today it must be. Currently all changes, markup or  
otherwise, affect the status of a translation. Ideas were explored to  
help solve this symptom but nothing great came about.

So we must change our process, but how? We have a few options:

1. Keep the current system, and simply change the revision handling
2. The same as (1) but with a few other tweaks
3. Totally rethink the system

I'm in favor of (3) but personally don't know the topic well enough to  
comfortably design it. My gut says move towards a gettext based system  
with po/pot files but again it's not something I've used personally  
but do know others have and like it, that other large manuals use it  
with success, and that many tools exist to manage this including po  
specific text editors. And because several projects already do this we  
could evaluate several and decide what would work best for us. And, of  
course steal their HOWTO information ;)

Please, all comments, complaints, questions and suggestions are  
welcome. Especially from translators!

Regards,
Philip

P.S. Please only reply to phpdoc@..., this initial email CCs  
all doc lists to notify everyone of the topic.

--
PHP Spanish Documentation Mailing List (http://php.net/manual/es/)
To unsubscribe, visit: http://www.php.net/unsub.php


--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Gabriel PREDA-2]

+1 for SVN


Gabriel PREDA
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Certified MySQL Associate
Zend Certified Engineer
Senior Web-Applications Developer
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
http://e-radical.ro/


On Fri, Apr 10, 2009 at 11:58 PM, Philip Olson <philip@...> wrote:

> Greetings doc geeks,
>
> We need to improve our translations system for various reasons:
>
> A. It's difficult to work with:
> Honestly I don't know how translators put up with the current system and
> think it helps contribute to dead translations. Many kudos go to the
> translators!
>
> B. A new VCS is forcing a change:
> One day soon the PHP project will move from CVS to a new VCS, likely either
> Subversion or Git. This presents a real problem because we rely on CVS
> revision numbers for tracking the translations status.
>
> C. Translating markup and related worries:
> Whether or not an XML markup change affects translations shouldn't be on
> our minds, but today it must be. Currently all changes, markup or otherwise,
> affect the status of a translation. Ideas were explored to help solve this
> symptom but nothing great came about.
>
> So we must change our process, but how? We have a few options:
>
> 1. Keep the current system, and simply change the revision handling
> 2. The same as (1) but with a few other tweaks
> 3. Totally rethink the system
>
> I'm in favor of (3) but personally don't know the topic well enough to
> comfortably design it. My gut says move towards a gettext based system with
> po/pot files but again it's not something I've used personally but do know
> others have and like it, that other large manuals use it with success, and
> that many tools exist to manage this including po specific text editors. And
> because several projects already do this we could evaluate several and
> decide what would work best for us. And, of course steal their HOWTO
> information ;)
>
> Please, all comments, complaints, questions and suggestions are welcome.
> Especially from translators!
>
> Regards,
> Philip
>
> P.S. Please only reply to phpdoc@..., this initial email CCs all
> doc lists to notify everyone of the topic.
>
> --
> PHP Romanian Documentation Mailing List (http://php.net/manual/ro/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

[philipo]

Everyone,

The question is not whether we move the documentation repositories to  
a new version control system (like svn), because the entire php.net  
will do that. The question here is how to better manage the  
translation process and ideally come up with a solution before that  
move. Well, we must, because the change forces us to.

One option is to use gettext. This could mean English is in DocBook  
and translations use po files that are generated and managed from that  
DocBook. Please think about this and other options. This would involve  
using CAT and other modern methods so have a look:

   - CAT: http://en.wikipedia.org/wiki/Computer-assisted_translation

The po (gettext) option seems to offer the most tools and flexibility,  
and other projects in the Open Source community (like KDE, Ubuntu) use  
it. I may sound set in using po but am not and am only researching  
ideas. Please discuss this and other ideas. And, volunteers are needed  
to help migrate to whatever new system we choose.

Regards,
Philip


--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Dimitris Glezos]

On Sat, Apr 11, 2009 at 8:32 PM, Philip Olson <philip@...> wrote:

> Everyone,
>
> The question is not whether we move the documentation repositories to a new
> version control system (like svn), because the entire php.net will do that.
> The question here is how to better manage the translation process and
> ideally come up with a solution before that move. Well, we must, because the
> change forces us to.
>
> One option is to use gettext. This could mean English is in DocBook and
> translations use po files that are generated and managed from that DocBook.
>
>
> The po (gettext) option seems to offer the most tools and flexibility, and other
> projects in the Open Source community (like KDE, Ubuntu) use it.

PO files are considered a standard way of doing translations, and
quite a few tools are available which provide good support for both
translators and developers. It allows the translators to work more
efficiently by translating exactly what is needed, and also allows
them to detect easily when the source file has been changed and update
their translations.

To manage translations, Fedora uses Transifex, a tool which I
co-develop. Transifex presents translators with statistics, allows
them to get the files and submit them back directly to the source
versioning system via an easy-to-use web interface. The actual
translation can be done either using a desktop application or a
web-based system.

  http://transifex.org/
  http://lwn.net/Articles/325311/
  https://translate.fedoraproject.org/tx/

I believe it's great to finally see PHP consider using the full power
of Docbook/i18n, which I suspect will have a big improvement on the
manual's translation completion and quality.

-δ



--
Dimitris Glezos
Jabber ID: glezos@..., GPG: 0xA5A04C3B
http://dimitris.glezos.com/

"He who gives up functionality for ease of use
loses both and deserves neither." (Anonymous)
--

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Leonardo Boshell-2]

2009/4/10 Philip Olson <philip@...>:
> Greetings doc geeks,

Hello Philip and everyone,


> We need to improve our translations system for various reasons: (..)

Agreed. No questions there :) and by the way, thank you for pushing
for improvements on this area.


> So we must change our process, but how? We have a few options:
>
> 1. Keep the current system, and simply change the revision handling
> 2. The same as (1) but with a few other tweaks
> 3. Totally rethink the system

I think replacing the tools/workflow we use to handle the translations
(whether we use .po files or not, etc.) is not that important at this
moment. It's clear that there's a lot of room for improvement here,
but the way I see it, there are a couple of more fundamental things to
take care of before implementing new, better ways to manage the
translations:

a) As Philip mentions, the revision handling is a very basic thing
that is taken for granted right now, but the time will come when we
can't rely on CVS's automatic handling of $Revision$ strings anymore.
This system could be emulated on different VCS using pre-commit hooks,
which most if not all VCS software offer in one form or another. Or
maybe we can come up with a new system, if it adds extra value to
implement a new revision mechanism. I can't think of anything special
in this regard, so right now I'd go for using the same style of rev
numbers (1.XX), and just figuring out the way to keep using it on
whatever VCS that comes along.

b) Now, as far as CATs go, I know there are lots of options out there,
some using .po as the format at the translation level, and interfacing
with DocBook/XML, so the documentation team keeps working in the
original format (phpbook). That's great and I've been looking at some
of these tools, such as OmegaT, or Gnome's intltool, which could be
used interchangeably by translators, along with any other tools people
feel comfortable with, once our documentation tree is in the right
shape.

When I say "in the right shape", I'm talking about one of the things I
know have been discussed several times before, but without a
definitive conclusion (as far as I can recall, if that's not the case,
please let me know). To properly use these external tools, we need to
fix the doc tree first, so that every .xml is a valid, stand-alone
document. I remember this being a hot topic in the days of livedocs,
and whenever things like producing PDF files were mentioned, but as I
said, I can't recall seeing a conclusive answer regarding this.
However, I haven't been following the phpdoc list for some time, maybe
this have been discussed some more and somebody has a plan? :)


So that's my point of view. Once these two tasks are out of the way, I
think then will be the time to discuss about new tools, CATs, .po
files, and so on.

Thanks,

Leonardo B.
Spanish translation team

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Hannes Magnusson-3]

On Sun, Apr 12, 2009 at 02:34, Leonardo Boshell <lboshell@...> wrote:
> please let me know). To properly use these external tools, we need to
> fix the doc tree first, so that every .xml is a valid, stand-alone
> document. I remember this being a hot topic in the days of livedocs,
> and whenever things like producing PDF files were mentioned, but as I
> said, I can't recall seeing a conclusive answer regarding this.

See http://php.markmail.org/message/4mluqwhso3jwabw5

PDFs are no problem, PhD can already generate PDF files.

-Hannes

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Leonardo Boshell-2]

2009/4/12 Hannes Magnusson <hannes.magnusson@...>:
>
> See http://php.markmail.org/message/4mluqwhso3jwabw5

Nice, thanks for the pointer.

Can you perhaps post a patch and/or detailed instructions to replicate
what you describe on that thread? I'd like to give it a try and see if
maybe there's something that can be done about the performance impact
when every file references the whole set of entities.

Thanks,
Leonardo

PS. I post here because I wasn't subscribed to the phpdoc list, but
feel free to reply on the original thread, I should get that message
now.

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[philipo]

> a) As Philip mentions, the revision handling is a very basic thing
> that is taken for granted right now, but the time will come when we
> can't rely on CVS's automatic handling of $Revision$ strings anymore.
> This system could be emulated on different VCS using pre-commit hooks,
> which most if not all VCS software offer in one form or another. Or
> maybe we can come up with a new system, if it adds extra value to
> implement a new revision mechanism. I can't think of anything special
> in this regard, so right now I'd go for using the same style of rev
> numbers (1.XX), and just figuring out the way to keep using it on
> whatever VCS that comes along.

I do not believe this is a major concern because the various projects  
that use po files typically have tools setup that deal with this. So,  
in this case, I think we'd simply choose a set of pre-existing tools  
and slightly modify to fit our needs. But, the problem you describe is  
serious for the initial migration as I fear it'd be so painful that  
many translations would start from scratch. However, hopefully we'd  
figure out a nice way to at least import all of the up-to-date  
translated content...

> b) Now, as far as CATs go, I know there are lots of options out there,
> some using .po as the format at the translation level, and interfacing
> with DocBook/XML, so the documentation team keeps working in the
> original format (phpbook). That's great and I've been looking at some
> of these tools, such as OmegaT, or Gnome's intltool, which could be
> used interchangeably by translators, along with any other tools people
> feel comfortable with, once our documentation tree is in the right
> shape.


Good point, and I did try a few tools that required such validation  
but am not sure if all do. However, Hannes is working on this and I  
think plans to post a wiki entry with the details of his findings for  
discussion.

Regards,
Philip

--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[philipo]

> To manage translations, Fedora uses Transifex, a tool which I
> co-develop. Transifex presents translators with statistics, allows
> them to get the files and submit them back directly to the source
> versioning system via an easy-to-use web interface. The actual
> translation can be done either using a desktop application or a
> web-based system.
>
>  http://transifex.org/
>  http://lwn.net/Articles/325311/
>  https://translate.fedoraproject.org/tx/
>
> I believe it's great to finally see PHP consider using the full power
> of Docbook/i18n, which I suspect will have a big improvement on the
> manual's translation completion and quality.

Looks like an interesting option and definitely one to consider. If we  
go this route, would you be willing to help us set it up and/or help  
migrate at least translated content that's up to date? Or offer advice  
for such a transition? Do you feel transifex would work well for the  
PHP project? Most likely it would only be used for the PHP Manual but  
it's possible in the future other php.net projects could implement  
translations through it like phd, php-src errors, a few www.php.net  
elements... anything is possible I suppose. So yeah, something like  
transifex that's geared to manage multiple projects could find itself  
fully utilized here.

Regards,
Philip



--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Benjamin-30]

transifex looks like an excellent alternative.

Philip on the strategy of user registration, group organization, are there
any new proposal?, that the current method works very slowly, maybe a lot of
bureaucracy

Regards,
Benjamin Gonzales
ZendFramework Spanish translation team
http://zfdes.com
http://codigolinea.com
 

> To manage translations, Fedora uses Transifex, a tool which I
> co-develop. Transifex presents translators with statistics, allows
> them to get the files and submit them back directly to the source
> versioning system via an easy-to-use web interface. The actual
> translation can be done either using a desktop application or a
> web-based system.
>
>  http://transifex.org/
>  http://lwn.net/Articles/325311/
>  https://translate.fedoraproject.org/tx/
>
> I believe it's great to finally see PHP consider using the full power
> of Docbook/i18n, which I suspect will have a big improvement on the
> manual's translation completion and quality.

Looks like an interesting option and definitely one to consider. If we  
go this route, would you be willing to help us set it up and/or help  
migrate at least translated content that's up to date? Or offer advice  
for such a transition? Do you feel transifex would work well for the  
PHP project? Most likely it would only be used for the PHP Manual but  
it's possible in the future other php.net projects could implement  
translations through it like phd, php-src errors, a few www.php.net  
elements... anything is possible I suppose. So yeah, something like  
transifex that's geared to manage multiple projects could find itself  
fully utilized here.

Regards,
Philip



--
PHP Spanish Documentation Mailing List (http://php.net/manual/es/)
To unsubscribe, visit: http://www.php.net/unsub.php


--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php

[philipo]

On 14 Apr 2009, at 08:50, Benjamin wrote:

> transifex looks like an excellent alternative.
>
> Philip on the strategy of user registration, group organization, are  
> there
> any new proposal?, that the current method works very slowly, maybe  
> a lot of
> bureaucracy

I'm not exactly sure what you mean here (please explain) but am unsure  
how exactly our php.net accounts would interface with a system like  
transifex, like if we'd also have users with only access to transifex.  
This topic is part of the research process.

I don't see a problem with the theory behind the current registration  
system which is:
  - If people have work to contribute, they are given account
  - If people want an account, they first have work to contribute
  - Doing work does not require an account, but leads to one
  - People introduce themselves into the community before being given  
an account

How exactly this is managed could likely be improved but there are  
plenty of people out there who simply want, for example, a @php.net  
address without doing anything except ask for it. And gaining commit  
access without showing capacity to commit decent work or be involved  
with the community doesn't feel right. Not that anyone is suggesting  
that, but it's something the current system deals with.

All ideas and thoughts are welcome and up for discussion although this  
is getting a bit off topic for this thread.

Regards,
Philip


--
PHP Italian Documentation Mailing List (http://php.net/manual/it/)
To unsubscribe, visit: http://www.php.net/unsub.php