OFBiz
  1. OFBiz
  2. OFBIZ-2008

Tools to extract messages from *Labels.xml files to gettext PO files and merge translation results back

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Later
    • Affects Version/s: Trunk
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Tools to extract messages from *Labels.xml files to gettext PO files and merge translation results back

      1. gettext_.patch
        40 kB
        Jeroen van der Wal
      2. src.rar
        9 kB
        Jeroen van der Wal
      3. PropertyTranslators.patch
        12 kB
        Adrian Crum
      4. Po2ofbiz.java
        9 kB
        Nemanja Simonovic
      5. Ofbiz2po.java
        5 kB
        Nemanja Simonovic

        Activity

        Hide
        Nemanja Simonovic added a comment -

        Extracts *Labels.xml to gettext PO files

        Show
        Nemanja Simonovic added a comment - Extracts *Labels.xml to gettext PO files
        Hide
        Nemanja Simonovic added a comment -

        Tool to merge back messgaes from PO files in ofbiz *Labels.xml.

        This tool depends on '#KEY: ...' comment lines generated by Ofbiz2po to merge back translations.

        Show
        Nemanja Simonovic added a comment - Tool to merge back messgaes from PO files in ofbiz *Labels.xml. This tool depends on '#KEY: ...' comment lines generated by Ofbiz2po to merge back translations.
        Hide
        Jacques Le Roux added a comment -

        Hi Nemanja ,

        I never used PO and gettext, what are the advantages over ou current XML files ?

        Show
        Jacques Le Roux added a comment - Hi Nemanja , I never used PO and gettext, what are the advantages over ou current XML files ?
        Hide
        Jacques Le Roux added a comment -

        From Nemanja on dev ML

        most translation memory tools can tools work with po files, eg. pootle
        (pootle is translation collaboration server), OmegaT, kbabel;
        those greatly improve translation speed

        Please Nemanja, use rather comments in Jira issue to allow to more easily follow the thread, thanks.

        Yes, I see now, you want to be able to collaborate. I think you are already able to do it using our format, subversion and Eclipse or an UTF8 capable editor like Scite. Of course this is maybe not as easy as using i18n specialized tools...

        I will think about where we could put your patches in OFBiz OOTB, of course any ideas from a commiter is also welcome

        Show
        Jacques Le Roux added a comment - From Nemanja on dev ML most translation memory tools can tools work with po files, eg. pootle (pootle is translation collaboration server), OmegaT, kbabel; those greatly improve translation speed Please Nemanja, use rather comments in Jira issue to allow to more easily follow the thread, thanks. Yes, I see now, you want to be able to collaborate. I think you are already able to do it using our format, subversion and Eclipse or an UTF8 capable editor like Scite. Of course this is maybe not as easy as using i18n specialized tools... I will think about where we could put your patches in OFBiz OOTB, of course any ideas from a commiter is also welcome
        Hide
        Jeroen van der Wal added a comment -

        There are currently 11739 labels to be translated from English to the desired language and that's going to be a hell of a job using an editor.

        Exporting the labels to the po format gives us the opportunity to host a collaboration environment for just the translations. A good example is the translation of Filezilla using a Pootle server: http://pootle.locamotion.org/projects/filezilla/

        Show
        Jeroen van der Wal added a comment - There are currently 11739 labels to be translated from English to the desired language and that's going to be a hell of a job using an editor. Exporting the labels to the po format gives us the opportunity to host a collaboration environment for just the translations. A good example is the translation of Filezilla using a Pootle server: http://pootle.locamotion.org/projects/filezilla/
        Hide
        Adrian Crum added a comment -

        I'm all for having tools that make translations easier, and I'm sure the attached patches help. The files are not suitable for committing however. They are not in the proper patch file format and there are no Apache headers. Also, it would be convenient if build.xml files were updated to turn these classes into ant tasks.

        In the meantime, anyone wanting to do translations can still download and use the files on their local copy.

        Show
        Adrian Crum added a comment - I'm all for having tools that make translations easier, and I'm sure the attached patches help. The files are not suitable for committing however. They are not in the proper patch file format and there are no Apache headers. Also, it would be convenient if build.xml files were updated to turn these classes into ant tasks. In the meantime, anyone wanting to do translations can still download and use the files on their local copy.
        Hide
        Jacques Le Roux added a comment -

        Jeroen, Nemanja,

        First of all, thanks for your initiative, it is very appreciated (at least by me ;o) !

        It's all about organisation at this stage. As Adrian outlined, we need to have proper patches. This is not the harder part, you just have to read carefully http://docs.ofbiz.org/display/OFBADMIN/OFBiz+Contributors+Best+Practices. But then (or even before) some questions arise

        • Do we really need to put them in OFBz ?
          • If yes where shoud we put these new features ?
        • Should we think about giving a way for translation to be done as easily as in FileZilla ?
          • If yes who (which organisation ?) will provide the infrastructure resources ?
            • This could maybe done on an ASF infra server. In such case we would need an intermediation from PMC (ok, at least Adrian and I are already aware).
          • Jeroen have you an idea of the work load needed to set a Pootle server for OFBiz ?
          • In case we get to the point we have a such server running, how will we get back the translations done in OFBiz ?
            • Could this be done in the 2 ways (upload-merge from OFBiz to Pootle server and vice versa) ? I suppose this is common questions around a Pootle server so please give us more explanations...

        I'm sure I forget some points here, please fill in

        Show
        Jacques Le Roux added a comment - Jeroen, Nemanja, First of all, thanks for your initiative, it is very appreciated (at least by me ;o) ! It's all about organisation at this stage. As Adrian outlined, we need to have proper patches. This is not the harder part, you just have to read carefully http://docs.ofbiz.org/display/OFBADMIN/OFBiz+Contributors+Best+Practices . But then (or even before) some questions arise Do we really need to put them in OFBz ? If yes where shoud we put these new features ? Should we think about giving a way for translation to be done as easily as in FileZilla ? If yes who (which organisation ?) will provide the infrastructure resources ? This could maybe done on an ASF infra server. In such case we would need an intermediation from PMC (ok, at least Adrian and I are already aware). Jeroen have you an idea of the work load needed to set a Pootle server for OFBiz ? In case we get to the point we have a such server running, how will we get back the translations done in OFBiz ? Could this be done in the 2 ways (upload-merge from OFBiz to Pootle server and vice versa) ? I suppose this is common questions around a Pootle server so please give us more explanations... I'm sure I forget some points here, please fill in
        Hide
        Adrian Crum added a comment -

        Jacques,

        I don't think we need to go that far. I see this being more like the JavaDoc ant task that we already have - it's something people can run on their local copy and they can do with it what they want.

        Maybe a simpler and easier strategy would be to have this converted to an ant task - giving users the ability to create *.po files. Then it's up to the users to set up a system to manage those files (SourceForge account, private SVN server, etc).

        Show
        Adrian Crum added a comment - Jacques, I don't think we need to go that far. I see this being more like the JavaDoc ant task that we already have - it's something people can run on their local copy and they can do with it what they want. Maybe a simpler and easier strategy would be to have this converted to an ant task - giving users the ability to create *.po files. Then it's up to the users to set up a system to manage those files (SourceForge account, private SVN server, etc).
        Hide
        Nemanja Simonovic added a comment -

        I did read Contributors Best Practices (i missed the topic about Apache headers, i thought JIRA will add them automagicly). I could not find
        appropriate place for file in OfBIZ hierarchy so I gave up creating a patch and posted plain files.

        I think we could put when with rest of import/export/migration tools for OfBIZ; if there aren't any now I think some will
        certainly appear as more people starts using OfBIZ.

        If tools are with OfBIZ (or in some other public repository) everyone can setup pootle server (or use some other tool)
        for translation he is making.

        Show
        Nemanja Simonovic added a comment - I did read Contributors Best Practices (i missed the topic about Apache headers, i thought JIRA will add them automagicly). I could not find appropriate place for file in OfBIZ hierarchy so I gave up creating a patch and posted plain files. I think we could put when with rest of import/export/migration tools for OfBIZ; if there aren't any now I think some will certainly appear as more people starts using OfBIZ. If tools are with OfBIZ (or in some other public repository) everyone can setup pootle server (or use some other tool) for translation he is making.
        Hide
        Adrian Crum added a comment -

        Probably the best place to put it is in the framework/base component. You'll have to modify the build.xml file in framework/base to create the jar(s) separately and copy them to the ofbiz home folder (so the task can be run from the ofbiz home folder).

        We could set up an import/export folder where you could drop the *.po files - and set the folder for SVN ignore. That will allow someone to check in/out from the folder into a separate *.po file repository.

        Each component's build.xml file would have two new targets - like "label-import-po" and "label-export-po" - to import/export the translated files.

        If that sounds like too much work, you could do the same thing as the WebTools XML file import/export capability. Just provide a UI in WebTools to import/export *.po files from/to a user-specified folder. OFBiz would have to be running to do the import/export if you take that route.

        Show
        Adrian Crum added a comment - Probably the best place to put it is in the framework/base component. You'll have to modify the build.xml file in framework/base to create the jar(s) separately and copy them to the ofbiz home folder (so the task can be run from the ofbiz home folder). We could set up an import/export folder where you could drop the *.po files - and set the folder for SVN ignore. That will allow someone to check in/out from the folder into a separate *.po file repository. Each component's build.xml file would have two new targets - like "label-import-po" and "label-export-po" - to import/export the translated files. If that sounds like too much work, you could do the same thing as the WebTools XML file import/export capability. Just provide a UI in WebTools to import/export *.po files from/to a user-specified folder. OFBiz would have to be running to do the import/export if you take that route.
        Hide
        Adrian Crum added a comment -

        The attached PropertyTranslators.patch file contains some classes that might be useful for property file translation.

        I coded this in a hurry and it is not tested. I put it here to demonstrate a more generic and extend-able approach to the problem.

        Show
        Adrian Crum added a comment - The attached PropertyTranslators.patch file contains some classes that might be useful for property file translation. I coded this in a hurry and it is not tested. I put it here to demonstrate a more generic and extend-able approach to the problem.
        Hide
        Jeroen van der Wal added a comment -

        I've taken the tools of Nemanja and updated it to have a fully GNU Gettext compatible solution for translating OFBiz.

        Translating OFbiz requires these steps:
        1. Run Export.java to export all labels from the *Labels.xml files
        2. Edit the .po file of your language using an gettext editor like poedit. If your desired lanugage does not exist copy the .pot file to a .po file with the appropriate language code.
        3. When done translating run Merge.java the merge the translated labels into the OFBiz .xml files.

        The code is still rough, and we will provide a patch to integrate it into webtools later.

        Show
        Jeroen van der Wal added a comment - I've taken the tools of Nemanja and updated it to have a fully GNU Gettext compatible solution for translating OFBiz. Translating OFbiz requires these steps: 1. Run Export.java to export all labels from the *Labels.xml files 2. Edit the .po file of your language using an gettext editor like poedit. If your desired lanugage does not exist copy the .pot file to a .po file with the appropriate language code. 3. When done translating run Merge.java the merge the translated labels into the OFBiz .xml files. The code is still rough, and we will provide a patch to integrate it into webtools later.
        Hide
        Jeroen van der Wal added a comment - - edited

        We worked on integrating the Gettext tools into OFBiz webtools. Here is what we changed:
        1. Put the Ofbiz2po and Po2ofbiz code into Adrians PropertyTranslator utility class.
        2. Replaced our file searching classes with the OFBiz utility methods, so now finding OFBiz label files is faster because we don't need to search the whole OFBiz folder recursively.
        3. Created a service and UI in OFBiz webtools for importing and exporting the labels to and from Gettext format.

        We noticed that some labels in productUiLabels.xml contain unescaped markup:

        <property key="ProductRedExplanation">
        <value xml:lang="de"><b style="color: red;">Rot</b> Datum/Zeit Angaben sind rot markiert wenn die aktuelle Zeit vor dem Von Datum oder nach dem Bis Datum ist. Wenn das Von Datum <b style="rot">rot</b> ist, ist die Zuordnung noch nicht aktiv; falls das Bis Datum <b style="color: red;">rot</b> ist, ist die Zuordnung abgelaufen (<u>und sollte vermutlich gelöscht werden</u>).</value>

        These labels are not exported to gettext format correctly and appear in the console and log for further analysis.

        Show
        Jeroen van der Wal added a comment - - edited We worked on integrating the Gettext tools into OFBiz webtools. Here is what we changed: 1. Put the Ofbiz2po and Po2ofbiz code into Adrians PropertyTranslator utility class. 2. Replaced our file searching classes with the OFBiz utility methods, so now finding OFBiz label files is faster because we don't need to search the whole OFBiz folder recursively. 3. Created a service and UI in OFBiz webtools for importing and exporting the labels to and from Gettext format. We noticed that some labels in productUiLabels.xml contain unescaped markup: <property key="ProductRedExplanation"> <value xml:lang="de"><b style="color: red;">Rot</b> Datum/Zeit Angaben sind rot markiert wenn die aktuelle Zeit vor dem Von Datum oder nach dem Bis Datum ist. Wenn das Von Datum <b style="rot">rot</b> ist, ist die Zuordnung noch nicht aktiv; falls das Bis Datum <b style="color: red;">rot</b> ist, ist die Zuordnung abgelaufen (<u>und sollte vermutlich gelöscht werden</u>).</value> These labels are not exported to gettext format correctly and appear in the console and log for further analysis.
        Hide
        Adrian Crum added a comment -

        Jeroen,

        Thank you very much for your continued work on this. The code I submitted was not tested - did you have a chance to try it out? Does it work properly?

        Regarding the unescaped markup example: there is an even bigger problem with that translation - it uses embedded HTML in a UI label. That is a bad design and should be corrected.

        If no one gets to this before me I will work on it. I'm trying to finish another issue first.

        Show
        Adrian Crum added a comment - Jeroen, Thank you very much for your continued work on this. The code I submitted was not tested - did you have a chance to try it out? Does it work properly? Regarding the unescaped markup example: there is an even bigger problem with that translation - it uses embedded HTML in a UI label. That is a bad design and should be corrected. If no one gets to this before me I will work on it. I'm trying to finish another issue first.

          People

          • Assignee:
            Bilgin Ibryam
            Reporter:
            Nemanja Simonovic
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development