Translatewiki.net uses oregano to deploy MediaWiki. Oregano is simple deployment tool. Oregano is documented in a blog post.
There is a script called
twn-update-all that updates MediaWiki core, extensions and composer dependencies and updates the localisation cache. It does not run schema changes (update.php). We run the master branch of MediaWiki and extensions. For this reason the deployer should be aware of the latest MediaWiki changes and whether they require configuration or schema changes. The script automatically uses
sudo -u betawiki and will fail if you don't have access.
The oregano commands must be run with
sudo -u betawiki.
oregano itself is in path and it must be run in the
/srv/mediawiki directory. It is recommended to add alias b for the sudo command. After deploying a new version, please monitor messages from Rakkaus in #mediawiki-i18n and be ready to rollback in case of issues. If you notice new PHP notices or warnings, those should be investigated and reported to Phabricator where applicable. You can also inspect the log file in
I (Nike) usually perform the following tasks during the regular deployment window:
- Check error logs before production update.
- Update packages: apt update, apt upgrade.
- Update production environment but do not deploy (do not create and release a tag). See deployment example below.
- Test on canary while checking error logs.
- If everything is OK, deploy and monitor error logs.
- Check #affects-translatewiki.net to see if issues are resolved (e.g. no more seen in logs), or need updating
Report errors as issues in Phabricator with #affects-translatewiki.net tag.
Full deployment example
cd /srv/mediawiki # Update twn-update-all twn-update-database # Test on canary # Deploy and release b oregano tag b oregano deploy
How to test code changes before deploying
It is possible to test code changes before deploying them to all users with
oregano. For example in Chrome it is done like this
- Go to translatewiki.net
- Open developer tools
- Open tab Application
- On the right column, under heading Storage, find item Cookies and expand it
- Open item https://translatewiki.net from the expanded list
- Find an empty row, right click on it, and choose Add new
- Write env as name, x as value, you can leave the rest as defaults
- Reload the page
You know that you did this correctly if you see orange background stripes and a "You are using a canary" message near the page title. This varies a bit depending on the skin and which page you are on, so for example on Timeless the colored stripes are only visible at the very bottom, and the message is not visible on our main page.
You can now test your changes. Once you are finished, do remember to delete the cookie.
sudo systemctl restart mw-jobrunner.
How to handle resolve conflicts during update
We occasionally cherry-pick patches or do local patches for various reasons. Although we try to keep those at minimum, updating the code may fail due to a merge conflict caused by some of these patches. You will notice this in the output of the command and often also L10N cache rebuild will fail because of fatal errors due to those conflicts.
Here is how to get rid of a conflicting patch (i.e. because an updated version was merged to master):
cd /srv/mediawiki/workdir/extensions/SomeExtension b git rebase --abort b git rebase -i # Drop the affected patches
Other times the patches might still be needed, in which case you need to solve the merge conflicts :(
How to rollback a deployment
If the deployment causes issues, such as bringing the site down, or some feature not working, or lot of log spam, it is recommended that you revert the deployment. This command reverts the site to the previously deployed version:
b oregano rollback
After the immediate issues are averted, it is time to investigate what went wrong, fix it, and think if this can be prevented in the future.
How to do a configuration change
- Submit a patch to the translatewiki repository in Gerrit.
- Get the patch merged. Self-merges are not forbidden, but discouraged.
- On translatewiki.net shell, run
twn-update-config. This will deploy the config change immediately.
- Test the change on the site and monitor the error log as above.
Optionally, you can also send a out a message to IRC to keep other admins in sync. It works like this
udpcast Your message here.
oreganoor you will bring the site down. Do note that
twn-update-allalso updates config.
- PrivateSettings.php and DatabaseSettings.php: Keys, passwords and other things which should not public. This file is only editable locally on the server.
- LanguageSettings.php and FallbackSettings.php: Settings related to configuration of available translation languages
- PermissionSettings.php: Most $wgGroupPermission things go here
- TranslateSettings.php: Settings related to the Translate extensions
- TranslatewikiSettings.php: Everything else
- Go to
cd /home/betawiki/configand edit the broken configuration with text editor (
b nano ...php, or
b git revert HEAD.
- Commit the working code locally with
b git commit -av
- Submit a fix to Gerrit and get it merged
b git fetchand
b git rebase -i origin/master– remove your fixup commit to avoid merge conflicts. If you still get conflicts,
b git rebase --abortshould help to get back to the previous state.
Updating a single extension
If you need to update one extension, for example UniversalLanguageSelector:
cd /srv/mediawiki/workdir/extensions/UniversalLanguageSelector # Make sure to go to the correct directory b git pull --rebase cd /srv/mediawiki/workdir php maintenance/rebuildLocalisationCache.php --threads=4 cd /srv/mediawiki b oregano tag b oregano deploy