Blueprint: Translation handling
|Proposal||Solve current translation handling restrictions|
|Status||Draft, Discussion, Voting Phase, Accepted, Declined, Withdrawn|
|Current Progress||Unknown, Started, Good Progress, Bad Progress, Stalled, Review Needed, DONE|
|Topic for Gerrit||translation|
- Started during TYPO3 CMS 7 development
- [DONE] All localization files got id's in 6.2 and 7, there is a travis-ci check to verify that
- [DONE] pootle translation now works on 6.2 git branch, 7 is currently disabled
- [DONE] It is allowed to move language files around in 7, id must be kept
Goals / Motivation
- Move pootle server from Dominic Feyer's private hosting to TYPO3 infrastructure
- Update pootle to latest version
- Enable moving XLF files around (in master branch, ...) while keeping the existing translations and not having to translate everything twice (currently the case)
- Enable removal of deprecated labels while still distributing them for older TYPO3 releases
- 10 man days for the infrastructure project: creating chef recipes, moving existing server to the new infrastructure, migrating Pootle
- 15 man days for supporting move of XLF files in TYPO3 Core with Pootle, adapting localization distribution scripts (targeting translated labels for a given TYPO3 version) and adapting supported TYPO3 releases to use the new distribution channel (version-wise) with a fallback to current scheme as last resort.
At the time of this writing, the translation handling of core .xlf files is suboptimal:
Localizations are identified by their filename, path and label identifier. Localization packages are identical for all versions. The translation server works on the core of latest master. So, if a locallang.xlf file is moved around in the core, the according translations from the old file locations - used in older core versions - will vanish on the translation server and the new file has to be translated again. A similar problem exists if single labels are removed from locallang.xlf files. Therefor, moving locallang files and removing single labels from files is currently not allowed in core development. This is unfortunate since it prohibits refactoring of localization related stuff and actively denies eg. merging two extensions into one.
- [DONE] Add a unique identifier to each localization file (a unique timestamp) that persists (is not changed) if a file is moved. Merge this patch to both master and 6.2. If this is done, moving of locallang files on master is allowed again.
<?xml version="1.0"> <xliff version="1.0" xmlns:t3="https://typo3.org/schemas/xliff"> <file t3:id="1415627320" ...
git log -p -M origin/TYPO3_6-2..master | grep ^rename | grep xlf
- [DONE] As immediate action, pootle will work on 6.2 core as base, not on master. This allows moving files in master, until scripting described below is done.
- Script pootle incoming files from core source: Based on identifier, pootle creates a "flat" list of translation files (per project, i.e. system extension), independent of their specific file location within the extension. Current existing translation data will be migrated to those locations. This way, files can be moved around without pootle loosing existing translations.
- Script pootle localization package creation: Based on identifier and core version, separate "per release" localization packages will be generated.
- Improve localization extension in 6.2 and 7 core to fetch localization packages from a sub path containing the core version.
- Currently, each translation file is generated as xml and xlf in the packages, xml can be removed for 6.2 and 7 then, 4.5 will not be affected.
Issues NOT solved with this blueprint
- There is still no concept for the removal of single labels within one label file. For now, labels that are rendered unused with younger core versions, must stay.
- The signature of existing labels must not be changed: Eg. it is still not allowed to add for example a %s parameter to an existing label - this still would break old relases. And, it is not allowed to change the full meaning of an existing label - if wording is changed, it should still work for old releases.
- On implementation failures, fetching localizations for core versions may fail.
- On upgrading failures, existing translations may get lost.