Translating:Commit MediaWiki core

The below documentation is for translatewiki.net shell users. This information may confuse less technical users. Please do not get confused, you do not have to worry about whatever is written on this page.

Commit arrangements

The following commit arrangements have been made to spread workload.

• Siebrand does not commit on even days of the month
• Raymond does not commit on odd days of the month

Warnings

• The below processes can be incompatible. Subsequent automated exports can overwrite previous results. You are advised to commit changes to core separately from commits to extensions.
• Remember: when in doubt: do not commit. Shoot suspicious changes first, ask questions later.

Regular messages (core and extensions)

1. make certain that translatewiki.net is running the latest trunk version of MediaWiki. Use bupdate
2. check the previous commits for MediaWiki from translatewiki.net with lastcommit. An export time should be mentioned in the commit message (in UTC).
3. calculate the difference between the previous export time and the current time in hours. Round up.
4. run bpmw <hours>. The export result will be at http://translatewiki.net/static/mediawiki-export.tgz
5. make certain that the MediaWiki instance you commit from is updated to the latest trunk version. You can use prepare-commits.cmd for this if you use Windows.
   1. dowload the exported updates to their proper locations
6. review the outcome and make fixes where needed (see advice on top of the page)
7. commit. See mwr:47296 for example commit message. Use "extensions" instead of "core" when committing changes for extensions. For full exports, always include the export time in your commit message (in UTC).

Things to look for

1. Translated namespaces (should be canonical), this is very common for messages like editpage
2. metadata-fields (translated list items)
3. Bad html
4. Extra links, formatting and everything that does not seem to belong there, like usage of parser functions
5. English strings
6. linkprefix .. easily broken

Backporting to stable

This section describes the process of backporting translations to earlier branches. Only update branches that are not end of life. Backports to stable should happen about once every two months, or just before a release. Usually the release manager will somehow show activity. Try to coordinate with the release manager so that you have time to perform these steps.

Updating 3 branches takes about 2 hours. Actual times can vary based on your level of experience.

Prerequisites

1. checked out version of MW1.13, 1.14, 1.15 branches with a valid LocalSettings.php (no running database required)
2. updated $changed arrays with incompatible messages for core-1.1[345] in /var/www/w/TranslateSettings.php. This step is extremely important. All messages with a changed meaning or changed parameters should be considered incompatible. Note the most recent version that was checked when updating the arrays in /var/www/w/TranslateSettings.php (outdated version).

Steps

1. run nice php export.php --target=$HOME/export/114 --lang=* --group=core-1.14 to export all languages for core-1.14. Export folder is arbitrary.
2. run a count of the current trunk messages using maintenance/language/countMessages.php. Make note of the count
3. download the export results (except for English)
4. fairly complete languages that were not yet part of a branch, can be added at your discretion. Also update RELEASE-NOTES and Names.php.
5. remove MessagesXx.php that are not part of the branch.
6. merge compatible changes to MessagesEn.php from trunk
7. rebuild all messages files with php rebuildLanguage.php --lang=all --remove-unknown (or whatever is available for the branch)
8. run a count of the current trunk messages using maintenance/language/countMessages.php. Calculate the number of added messages.
9. commit. See mwr:47278 for example commit message. This can cause huge commit mails (5MB+ is not unusual).

Backporting translations for other message groups is not (yet) available.

Extension special page aliases

Exporting special page aliases for extensions is part of bpmw. Extension special page aliases can be exported and updated automatically. Because of this, it is most efficient to just export everything regularly.

• run exportsp
• review the outcome and make fixes where needed (see advice on top of the page)
• commit. See mwr:42268 for example commit message. Alternatively commit with a regular extension localisation export run.

Magic words, special pages aliases

Needs updating now that mwcore-export.php exists.

• Requests for commit are made on Translating:Tasks.
• Get your input from Special:Magic for the a group/language combination by pressing Export
• Be alert that backwards incompatible changes in everything else can lead to brokenness in wikis. Review changes with extra care.

Namespaces

Needs updating now that mwcore-export.php exists.

• Requests for commit are made on Translating:Tasks.
• Get your input from Special:Magic for the a group/language combination by pressing Export
• If existing namespaces are changed, make certain that you add them as namespace aliases. See f.e. MessagesFr.php for an example ($namespaceAliases)

Command line scripts related to committing messages

/usr/local/bin/exportsp 

Export special page aliases for all defined extensions (see defined in Translate/aliases.txt)

php /var/www/w/extensions/Translate/scripts/alias-export.php --target=$HOME/export/ --lang=*

/usr/local/bin/bclean 

Cleanup your personal export folder

rm -rf $HOME/export/*
rm -f /var/www/static/mediawiki-export.tgz

/usr/local/bin/bx <group> <langs> (also bexport) 

Export multiple languages (langs as f.e. "nl,de,fr") for a group (group as f.e. "ext-cite"). Result is written to folder ~/export.

cd /var/www/w/extensions/Translate/scripts
php export.php --target=$HOME/export --group=$1 --lang="$2"

/usr/local/bin/bprep <hours>

Prepare an export script for all changed languages in the last <hours> hours as executable export.sh in your export folder.

cd /var/www/w/extensions/Translate/scripts
mkdir -p $HOME/export
php autoexport.php --summarize --skip=test,files,en,got,gan --format='bx $GROUP $LANG' --hours=$1 > $HOME/export/export.sh
chmod +x $HOME/export/export.sh

/usr/local/bin/bupdate 

Update translatewiki.net to the latest version of MediaWiki. Watch for "C" (assure you have reviewed localisation related commits, and watch for "C" in the output, meaning "Conflict"). If merge conflicts occur, place back conflictedfile.mine and resolve the conflict, or warn another shell user. Executing this command is announced on #mediawiki-i18n. Fuzzy (first using fuzzy [message], then using fuzzyr [message] all messages that need fuzzying immediately after updating. Be careful: fuzzyr will not allow spaces in the reason or the message key. You will do wisely to use underscores and a trailing slash for the key, and underscores in the reason.

#!/bin/sh
 
WIKI=/var/www/w
WIKINAME=translatewiki.net
 
udpcast updating $WIKINAME...
 
svn up $WIKI
cd $WIKI/extensions/Translate/
php $WIKI/extensions/Translate/scripts/createMessageIndex.php --quiet || udpcast message index creation failed &
make -s -C $WIKI/serialized || udpcast serialisation of language files failed &
wait
REV=`svn info $WIKI | grep ^Revision | cut -d" " -f2`
udpcast updated $WIKINAME to r$REV

/usr/local/bin/fuzzyr <messagekey> [comment]

Mark all translations for <messagekey> as fuzzy with [comment].

# fuzzy for real.
# $1 message ID
# $2 reason
cd /var/www/w/extensions/Translate/scripts
php fuzzy.php --skiplanguages=en,test --really $1 --comment=$2

sync-extmaintained <group> 

Synchronised some known externally maintained languages for <group>. Conflicts should be dealt with manually.

# Siebrand / 2008-07-13
# sync externally maintained languages to local database
cd /var/www/w/extensions/Translate/scripts
/usr/bin/nice php sync-group.php --group=$1 --lang=en,fa,he,yue,zh-hans,zh-hant,lzh,qqq