Mercurial > kallithea
changeset 8417:a564e87a3958 stable
docs: move i18n how_to directly under docs
author | Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> |
---|---|
date | Wed, 06 May 2020 19:52:05 +0200 |
parents | 5730b30b1ad4 |
children | 84fd6a8d38df |
files | docs/dev/i18n.rst docs/dev/translation.rst docs/index.rst kallithea/i18n/how_to |
diffstat | 4 files changed, 258 insertions(+), 258 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/dev/i18n.rst Wed May 06 19:52:05 2020 +0200 @@ -0,0 +1,257 @@ +.. _translations: + +============ +Translations +============ + +Translations are available on Hosted Weblate at the following URL: + + https://hosted.weblate.org/projects/kallithea/kallithea/ + +Registered users may contribute to the existing languages, or request a new +language translation. + + +Translating using Weblate +------------------------- + +Weblate_ offers a simple and easy to use interface featuring glossary, machine +translation, suggestions based on similar translations in other projects, +automatic checks etc. Weblate imports the source code tree directly from +the version control system, and commits edits back from time to time. + +When registering at Weblate, make sure you use the name and email address you +prefer to be used when your changes are committed. We can and probably will +amend changesets coming from Weblate, but having things right from the beginning +makes things easier. + +Weblate performs sanity checks all the time and tries to prevent you from ignoring +them. Most common mistakes are inconsistent punctuation, whitespace, missing or extra +format parameters, untranslated strings copied into the translation. Please perform +necessary corrections when they're needed, or override the false positives. + + +Merging translations from Weblate (admin-only) +---------------------------------------------- + +Weblate rebases its changes every time it pulls from our repository. Pulls are triggered +by a web hook from Our Own Kallithea every time it receives new commits. Usually merging +the new translations is a straightforward process consisting of a pull from the Weblate-hosted +repository which is available under the Data Exports tab in the Weblate interface. + +Weblate tries to minimise the number of commits, but that doesn't always work, especially +when two translators work with different languages at more or less the same time. +It makes sense sometimes to re-order or fold commits by the same author when they touch +just the same language translation. That, however, may confuse Weblate sometimes, in +which case it should be manually convinced it has to discard the commits it created by +using its administrative interface. + + +Regenerating translations after source code changes (admin-only) +---------------------------------------------------------------- + +When the Kallithea source code changes, both the location as the content of +translation strings can change. It is therefore necessary to regularly +regenerate the `kallithea.pot` file containing these strings, as well as aligning +the translation files (`*.po`). + +First update the translation strings:: + + python3 setup.py extract_messages + +Then regenerate the translation files. This could either be done with `python3 +setup.py update_catalog` or with `msgmerge` from the `gettext` package. As +Weblate is also touching these translation files, it is preferred to use the +same tools (`msgmerge`) and settings as Weblate to minimize the diff:: + + find kallithea/i18n -name kallithea.po | xargs -I '{}' \ + msgmerge --width=76 --backup=none --previous --update '{}' \ + kallithea/i18n/kallithea.pot + + +Manual creation of a new language translation +--------------------------------------------- + +In the prepared development environment, run the following to ensure +all translation strings are extracted and up-to-date:: + + python3 setup.py extract_messages + +Create new language by executing following command:: + + python3 setup.py init_catalog -l <new_language_code> + +This creates a new translation under directory `kallithea/i18n/<new_language_code>` +based on the translation template file, `kallithea/i18n/kallithea.pot`. + +Edit the new PO file located in `LC_MESSAGES` directory with poedit or your +favorite PO files editor. After you finished with the translations, check the +translation file for errors by executing:: + + msgfmt -f -c kallithea/i18n/<new_language_code>/LC_MESSAGES/<updated_file.po> + +Finally, compile the translations:: + + python3 setup.py compile_catalog -l <new_language_code> + + +Manually updating translations +------------------------------ + +Extract the latest versions of strings for translation by running:: + + python3 setup.py extract_messages + +Update the PO file by doing:: + + python3 setup.py update_catalog -l <new_language_code> + +Edit the newly updated translation file. Repeat all steps after the +`init_catalog` step from the 'new translation' instructions above. + + +Testing translations +-------------------- + +Edit `kallithea/tests/conftest.py` and set `i18n.lang` to `<new_language_code>` +and run Kallithea tests by executing:: + + py.test + + +Managing translations with scripts/i18n tooling +----------------------------------------------- + +The general idea with the ``scripts/i18n`` tooling is to keep changes in the +main repository focussed on actual and reviewable changes with minimal noise. +Noisy generated or redundant localization changes (that are useful when +translations) are contained in the ``kallithea-i18n`` repo on the ``i18n`` +branch. The translation files in the main repository have no line numbers, no +untranslated entries, no fuzzy entries, no unused entries, and no constantly +changing records of "latest" this and that (name, date, version, etc). + +The branches in the main repo (``default`` and ``stable``) will thus only have +stripped ``.pot`` and ``.po`` files: an (almost) empty +``kallithea/i18n/kallithea.pot`` file, and minimal ``.po`` files. There are no +binary ``.mo`` files in any repo - these are only generated when packaging for +release (or locally if installing from source). + +Generally, ``kallithea/i18n/`` should not be changed on the ``default`` and +``stable`` branches at all. The ``i18n`` branch should *only* change +``kallithea/i18n/`` . If there are changesets with exceptions from that, these +changesets should probably be grafted/redone in the "right" place. + +The basic flow is thus: + +0. All weblate translation is done on the ``i18n`` branch which generally is + based on the ``stable`` branch. +1. Graft the essential part of all new changes on the ``i18n`` branch to + ``stable`` (while normalizing to current stripped state of stable). +2. Merge from ``stable`` to ``i18n`` (while normalizing to the resulting + unstripped and fully ``msgmerge``'d state and ``.pot``-updating state). +3. Verify that the content of the ``i18n`` branch will give exactly the content + of the ``stable`` branch after stripping. If there is a diff, something has + to be fixed in one way or the other ... and the whole process should + probably be redone. + +Translate +^^^^^^^^^ + +First land full translation changes in the ``kallithea-i18n`` repo on the +``i18n`` branch. That can be done in pretty much any way you want. If changes +for some reason have to be grafted or merged, there might be odd conflicts due +to all the noise. Conflicts on the full ``i18n`` branch can perhaps be resolved +more easily using non-stripping normalization before merging:: + + python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot -r . + hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot $local $base $other $output' + +Land in main repository - stripped +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When the full i18n changes have landed on the ``i18n`` branch, prepare to land +them on ``stable``:: + + hg up -cr stable + python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot + +Consider all new ``i18n`` changes since last merge from ``stable``:: + + hg log -G --style compact -r 'only("i18n", children(::stable))' + +Graft them one by one (or in collapsed chunks) while normalizing. + +If the graft has conflicts, use the ``scripts/i18n`` normalization tool to +apply ``msgmerge`` and strip before doing 3-way merge and resolving conflicts:: + + hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot --strip $local $base $other $output' + +When all conflicts have been resolved, continue the graft:: + + hg graft --continue + +Then make sure any non-conflicting files are normalized and stripped too:: + + scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po + hg ci --amend --config ui.editor=true + +When things have been grafted to the ``stable`` branch, clean up history if +necessary: clean up the author and commit message when necessary, and perhaps +merge multiple changesets from same contributor. + +Merge back to ``i18n`` +^^^^^^^^^^^^^^^^^^^^^^ + +For any i18n changes that for some reason have been done on the ``stable`` +branch, apply them manually on the ``i18n`` branch too - perhaps by grafting +and editing manually. The merge done in this step will `not` take care of it. +If the verification step done a bit later points out that something has been +missed, strip and go back to this point. + +Then merge back to the ``i18n`` branch using normalization while keeping the +full ``.po`` files, and updating the full ``.pot`` and ``.po`` to current +state:: + + hg up -cr i18n + hg merge stable --tool internal:fail + hg revert kallithea/i18n/*/LC_MESSAGES/*.po -r . + hg resolve -m kallithea/i18n/*/LC_MESSAGES/*.po + hg resolve -l # verify all conflicts have been resolved + python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot + scripts/i18n normalize-po-files --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po + hg commit # "Merge from stable" + +Note: ``normalize-po-files`` can also pretty much be done manually with:: + + for po in kallithea/i18n/*/LC_MESSAGES/kallithea.po; do msgmerge --width=76 --backup=none --previous --update $po full.pot ; done + +Note: Additional merges from ``stable`` to ``i18n`` can be done any time. + +Verify +^^^^^^ + +Verify things are in sync between the full ``i18n`` branch and the stripped +``stable`` branch:: + + hg up -cr stable + hg revert -a -r i18n + python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot + scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po + hg diff + +If there is a diff, figure out where it came from, go back and fix the root +cause, and redo the graft/merge. + +Push +^^^^ + +The changes on the ``stable`` branch should now be ready for pushing - verify +the actual changes with a thorough review of:: + + hg out -pvr stable + +When ``stable`` changes have been pushed, also push the ``i18n`` branch to the +``kallithea-i18n`` repo so Weblate can see it. + + +.. _Weblate: http://weblate.org/
--- a/docs/dev/translation.rst Mon May 04 18:27:36 2020 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -.. _translations: -.. include:: ./../../kallithea/i18n/how_to
--- a/docs/index.rst Mon May 04 18:27:36 2020 +0200 +++ b/docs/index.rst Wed May 06 19:52:05 2020 +0200 @@ -74,7 +74,7 @@ :maxdepth: 1 contributing - dev/translation + dev/i18n dev/dbmigrations
--- a/kallithea/i18n/how_to Mon May 04 18:27:36 2020 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,255 +0,0 @@ -============ -Translations -============ - -Translations are available on Hosted Weblate at the following URL: - - https://hosted.weblate.org/projects/kallithea/kallithea/ - -Registered users may contribute to the existing languages, or request a new -language translation. - - -Translating using Weblate -------------------------- - -Weblate_ offers a simple and easy to use interface featuring glossary, machine -translation, suggestions based on similar translations in other projects, -automatic checks etc. Weblate imports the source code tree directly from -the version control system, and commits edits back from time to time. - -When registering at Weblate, make sure you use the name and email address you -prefer to be used when your changes are committed. We can and probably will -amend changesets coming from Weblate, but having things right from the beginning -makes things easier. - -Weblate performs sanity checks all the time and tries to prevent you from ignoring -them. Most common mistakes are inconsistent punctuation, whitespace, missing or extra -format parameters, untranslated strings copied into the translation. Please perform -necessary corrections when they're needed, or override the false positives. - - -Merging translations from Weblate (admin-only) ----------------------------------------------- - -Weblate rebases its changes every time it pulls from our repository. Pulls are triggered -by a web hook from Our Own Kallithea every time it receives new commits. Usually merging -the new translations is a straightforward process consisting of a pull from the Weblate-hosted -repository which is available under the Data Exports tab in the Weblate interface. - -Weblate tries to minimise the number of commits, but that doesn't always work, especially -when two translators work with different languages at more or less the same time. -It makes sense sometimes to re-order or fold commits by the same author when they touch -just the same language translation. That, however, may confuse Weblate sometimes, in -which case it should be manually convinced it has to discard the commits it created by -using its administrative interface. - - -Regenerating translations after source code changes (admin-only) ----------------------------------------------------------------- - -When the Kallithea source code changes, both the location as the content of -translation strings can change. It is therefore necessary to regularly -regenerate the `kallithea.pot` file containing these strings, as well as aligning -the translation files (`*.po`). - -First update the translation strings:: - - python3 setup.py extract_messages - -Then regenerate the translation files. This could either be done with `python3 -setup.py update_catalog` or with `msgmerge` from the `gettext` package. As -Weblate is also touching these translation files, it is preferred to use the -same tools (`msgmerge`) and settings as Weblate to minimize the diff:: - - find kallithea/i18n -name kallithea.po | xargs -I '{}' \ - msgmerge --width=76 --backup=none --previous --update '{}' \ - kallithea/i18n/kallithea.pot - - -Manual creation of a new language translation ---------------------------------------------- - -In the prepared development environment, run the following to ensure -all translation strings are extracted and up-to-date:: - - python3 setup.py extract_messages - -Create new language by executing following command:: - - python3 setup.py init_catalog -l <new_language_code> - -This creates a new translation under directory `kallithea/i18n/<new_language_code>` -based on the translation template file, `kallithea/i18n/kallithea.pot`. - -Edit the new PO file located in `LC_MESSAGES` directory with poedit or your -favorite PO files editor. After you finished with the translations, check the -translation file for errors by executing:: - - msgfmt -f -c kallithea/i18n/<new_language_code>/LC_MESSAGES/<updated_file.po> - -Finally, compile the translations:: - - python3 setup.py compile_catalog -l <new_language_code> - - -Manually updating translations ------------------------------- - -Extract the latest versions of strings for translation by running:: - - python3 setup.py extract_messages - -Update the PO file by doing:: - - python3 setup.py update_catalog -l <new_language_code> - -Edit the newly updated translation file. Repeat all steps after the -`init_catalog` step from the 'new translation' instructions above. - - -Testing translations --------------------- - -Edit `kallithea/tests/conftest.py` and set `i18n.lang` to `<new_language_code>` -and run Kallithea tests by executing:: - - py.test - - -Managing translations with scripts/i18n tooling ------------------------------------------------ - -The general idea with the ``scripts/i18n`` tooling is to keep changes in the -main repository focussed on actual and reviewable changes with minimal noise. -Noisy generated or redundant localization changes (that are useful when -translations) are contained in the ``kallithea-i18n`` repo on the ``i18n`` -branch. The translation files in the main repository have no line numbers, no -untranslated entries, no fuzzy entries, no unused entries, and no constantly -changing records of "latest" this and that (name, date, version, etc). - -The branches in the main repo (``default`` and ``stable``) will thus only have -stripped ``.pot`` and ``.po`` files: an (almost) empty -``kallithea/i18n/kallithea.pot`` file, and minimal ``.po`` files. There are no -binary ``.mo`` files in any repo - these are only generated when packaging for -release (or locally if installing from source). - -Generally, ``kallithea/i18n/`` should not be changed on the ``default`` and -``stable`` branches at all. The ``i18n`` branch should *only* change -``kallithea/i18n/`` . If there are changesets with exceptions from that, these -changesets should probably be grafted/redone in the "right" place. - -The basic flow is thus: - -0. All weblate translation is done on the ``i18n`` branch which generally is - based on the ``stable`` branch. -1. Graft the essential part of all new changes on the ``i18n`` branch to - ``stable`` (while normalizing to current stripped state of stable). -2. Merge from ``stable`` to ``i18n`` (while normalizing to the resulting - unstripped and fully ``msgmerge``'d state and ``.pot``-updating state). -3. Verify that the content of the ``i18n`` branch will give exactly the content - of the ``stable`` branch after stripping. If there is a diff, something has - to be fixed in one way or the other ... and the whole process should - probably be redone. - -Translate -^^^^^^^^^ - -First land full translation changes in the ``kallithea-i18n`` repo on the -``i18n`` branch. That can be done in pretty much any way you want. If changes -for some reason have to be grafted or merged, there might be odd conflicts due -to all the noise. Conflicts on the full ``i18n`` branch can perhaps be resolved -more easily using non-stripping normalization before merging:: - - python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot -r . - hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot $local $base $other $output' - -Land in main repository - stripped -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When the full i18n changes have landed on the ``i18n`` branch, prepare to land -them on ``stable``:: - - hg up -cr stable - python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot - -Consider all new ``i18n`` changes since last merge from ``stable``:: - - hg log -G --style compact -r 'only("i18n", children(::stable))' - -Graft them one by one (or in collapsed chunks) while normalizing. - -If the graft has conflicts, use the ``scripts/i18n`` normalization tool to -apply ``msgmerge`` and strip before doing 3-way merge and resolving conflicts:: - - hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot --strip $local $base $other $output' - -When all conflicts have been resolved, continue the graft:: - - hg graft --continue - -Then make sure any non-conflicting files are normalized and stripped too:: - - scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po - hg ci --amend --config ui.editor=true - -When things have been grafted to the ``stable`` branch, clean up history if -necessary: clean up the author and commit message when necessary, and perhaps -merge multiple changesets from same contributor. - -Merge back to ``i18n`` -^^^^^^^^^^^^^^^^^^^^^^ - -For any i18n changes that for some reason have been done on the ``stable`` -branch, apply them manually on the ``i18n`` branch too - perhaps by grafting -and editing manually. The merge done in this step will `not` take care of it. -If the verification step done a bit later points out that something has been -missed, strip and go back to this point. - -Then merge back to the ``i18n`` branch using normalization while keeping the -full ``.po`` files, and updating the full ``.pot`` and ``.po`` to current -state:: - - hg up -cr i18n - hg merge stable --tool internal:fail - hg revert kallithea/i18n/*/LC_MESSAGES/*.po -r . - hg resolve -m kallithea/i18n/*/LC_MESSAGES/*.po - hg resolve -l # verify all conflicts have been resolved - python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot - scripts/i18n normalize-po-files --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po - hg commit # "Merge from stable" - -Note: ``normalize-po-files`` can also pretty much be done manually with:: - - for po in kallithea/i18n/*/LC_MESSAGES/kallithea.po; do msgmerge --width=76 --backup=none --previous --update $po full.pot ; done - -Note: Additional merges from ``stable`` to ``i18n`` can be done any time. - -Verify -^^^^^^ - -Verify things are in sync between the full ``i18n`` branch and the stripped -``stable`` branch:: - - hg up -cr stable - hg revert -a -r i18n - python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot - scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po - hg diff - -If there is a diff, figure out where it came from, go back and fix the root -cause, and redo the graft/merge. - -Push -^^^^ - -The changes on the ``stable`` branch should now be ready for pushing - verify -the actual changes with a thorough review of:: - - hg out -pvr stable - -When ``stable`` changes have been pushed, also push the ``i18n`` branch to the -``kallithea-i18n`` repo so Weblate can see it. - - -.. _Weblate: http://weblate.org/