changeset 8382:fa373bf186b2 stable

scripts/i18n: update i18n howto with recipe for use of scripts/i18n tooling For now, just adding a new section. It should be integrated with the existing documentation. Some steps can perhaps be simplified or automated so we need less words.
author Mads Kiilerich <mads@kiilerich.com>
date Mon, 13 Apr 2020 22:49:07 +0200
parents 3c53e19719cd
children f7307da42aba
files kallithea/i18n/how_to
diffstat 1 files changed, 135 insertions(+), 0 deletions(-) [+]
line wrap: on
line diff
--- a/kallithea/i18n/how_to	Thu Apr 23 11:02:07 2020 +0200
+++ b/kallithea/i18n/how_to	Mon Apr 13 22:49:07 2020 +0200
@@ -117,4 +117,139 @@
     py.test
 
 
+Managing translations with scripts/i18n tooling
+-----------------------------------------------
+
+The general idea with the ``scripts/i18n`` tooling is to keep changes in the
+main repository focussed on actual and reviewable changes with minimal noise.
+Noisy generated or redundant localization changes (that are useful when
+translations) are contained in the ``kallithea-i18n`` repo on the ``i18n``
+branch. The translation files in the main repository have no line numbers, no
+untranslated entries, no fuzzy entries, no unused entries, and no constantly
+changing records of "latest" this and that (name, date, version, etc).
+
+The branches in the main repo (``default`` and ``stable``) will thus only have
+stripped ``.pot`` and ``.po`` files: an (almost) empty
+``kallithea/i18n/kallithea.pot`` file, and minimal ``.po`` files. There are no
+binary ``.mo`` files in any repo - these are only generated when packaging for
+release (or locally if installing from source).
+
+Generally, ``kallithea/i18n/`` should not be changed on the ``default`` and
+``stable`` branches at all. The ``i18n`` branch should *only* change
+``kallithea/i18n/`` . If there are changesets with exceptions from that, these
+changesets should probably be grafted/redone in the "right" place.
+
+The basic flow is thus:
+
+0. All weblate translation is done on the ``i18n`` branch which generally is
+   based on the ``stable`` branch.
+1. Graft the essential part of all new changes on the ``i18n`` branch to
+   ``stable`` (while normalizing to current stripped state of stable).
+2. Merge from ``stable`` to ``i18n`` (while normalizing to the resulting
+   unstripped and fully ``msgmerge``'d state and ``.pot``-updating state).
+3. Verify that the content of the ``i18n`` branch will give exactly the content
+   of the ``stable`` branch after stripping. If there is a diff, something has
+   to be fixed in one way or the other ... and the whole process should
+   probably be redone.
+
+Translate
+^^^^^^^^^
+
+First land full translation changes in the ``kallithea-i18n`` repo on the
+``i18n`` branch. That can be done in pretty much any way you want. If changes
+for some reason have to be grafted or merged, there might be odd conflicts due
+to all the noise. Conflicts on the full ``i18n`` branch can perhaps be resolved
+more easily using non-stripping normalization before merging::
+
+  python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot -r .
+  hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot $local $base $other $output'
+
+Land in main repository - stripped
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the full i18n changes have landed on the ``i18n`` branch, prepare to land
+them on ``stable``::
+
+  hg up -cr stable
+  python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot
+
+Consider all new ``i18n`` changes since last merge from ``stable``::
+
+  hg log -G --style compact -r 'only("i18n", children(::stable))'
+
+Graft them one by one (or in collapsed chunks) while normalizing.
+
+If the graft has conflicts, use the ``scripts/i18n`` normalization tool to
+apply ``msgmerge`` and strip before doing 3-way merge and resolving conflicts::
+
+  hg resolve kallithea/i18n/ --tool X --config merge-tools.X.executable=python3 --config merge-tools.X.args='scripts/i18n normalized-merge --merge-pot-file full.pot --strip $local $base $other $output'
+
+When all conflicts have been resolved, continue the graft::
+
+  hg graft --continue
+
+Then make sure any non-conflicting files are normalized and stripped too::
+
+  scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po
+  hg ci --amend --config ui.editor=true
+
+When things have been grafted to the ``stable`` branch, clean up history if
+necessary: clean up the author and commit message when necessary, and perhaps
+merge multiple changesets from same contributor.
+
+Merge back to ``i18n``
+^^^^^^^^^^^^^^^^^^^^^^
+
+For any i18n changes that for some reason have been done on the ``stable``
+branch, apply them manually on the ``i18n`` branch too - perhaps by grafting
+and editing manually. The merge done in this step will `not` take care of it.
+If the verification step done a bit later points out that something has been
+missed, strip and go back to this point.
+
+Then merge back to the ``i18n`` branch using normalization while keeping the
+full ``.po`` files, and updating the full ``.pot`` and ``.po`` to current
+state::
+
+  hg up -cr i18n
+  hg merge stable --tool internal:fail
+  hg revert kallithea/i18n/*/LC_MESSAGES/*.po -r .
+  hg resolve -m kallithea/i18n/*/LC_MESSAGES/*.po
+  hg resolve -l  # verify all conflicts have been resolved
+  python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot
+  scripts/i18n normalize-po-files --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po
+  hg commit  # "Merge from stable"
+
+Note: ``normalize-po-files`` can also pretty much be done manually with::
+
+  for po in kallithea/i18n/*/LC_MESSAGES/kallithea.po; do msgmerge --width=76 --backup=none --previous --update $po full.pot ; done
+
+Note: Additional merges from ``stable`` to ``i18n`` can be done any time.
+
+Verify
+^^^^^^
+
+Verify things are in sync between the full ``i18n`` branch and the stripped
+``stable`` branch::
+
+  hg up -cr stable
+  hg revert -a -r i18n
+  python3 setup.py extract_messages && cp kallithea/i18n/kallithea.pot full.pot && hg revert kallithea/i18n/kallithea.pot
+  scripts/i18n normalize-po-files --strip --merge-pot-file full.pot kallithea/i18n/*/LC_MESSAGES/kallithea.po
+  hg diff
+
+If there is a diff, figure out where it came from, go back and fix the root
+cause, and redo the graft/merge.
+
+Push
+^^^^
+
+The changes on the ``stable`` branch should now be ready for pushing - verify
+the actual changes with a thorough review of::
+
+  hg out -pvr stable
+
+When ``stable`` changes have been pushed, also push the ``i18n`` branch to the
+``kallithea-i18n`` repo so Weblate can see it.
+
+
 .. _Weblate: http://weblate.org/