{{support}} Adding TheWikipediaLibrary Card Platform TranslateWiki

{{support}} Adding TheWikipediaLibrary Card Platform TranslateWiki

I've been working on TheWikipediaLibrary Card Platform https://wikipedialibrary.wmflabs.org/ for some time now, and I think we have the codebase in a place where it's ready for localization.

Message documentation:

Since this project is a Django app, which has some nice tooling around gnu gettext, we decided to bake our message documentation into our code comments and let gettext add it to the qqq po file. https://github.com/WikipediaLibrary/TWLight/blob/master/locale/qqq/LC_MESSAGES/django.po

Adding new message translations:

In addition to adding new translation files, the universe of available languages in the user preferences should be expanded by updating the LANGUAGES variable in our base settings. https://github.com/WikipediaLibrary/TWLight/blob/master/TWLight/settings/base.py A nightly build script pulls the latest changes from master and applies them so long as the changes don't fail the unit tests.

Limitations:

This app has content that lives in the database. That content is comprised of fairly large blobs of text that are not visible to Translatewiki via the code repo. We're on the hook for translation of that content, but I don't really see a good way around that. We do have some folks internal to our project willing and able to expand translation for that content. That's how we got the Finnish and French that we currently have in place.

Jsn.sherman (talk)11:32, 1 November 2017

You need more space for the login button. Finnish text is truncated.

> In addition to adding new translation files, the universe of available languages in the user preferences should be expanded by updating the LANGUAGES

Can you do that? We have no support for such automation (nor even manual process) right now.

Please consider using https://github.com/wikimedia/jquery.uls for language selection, or at least take the language names from https://github.com/wikimedia/language-data or CLDR. We don't want language names translated here again and again for different projects.

Nike (talk)11:59, 1 November 2017
 

Also, is there a reason you are using the non-standard code iw for Hebrew instead of the standard one he?

Nike (talk)10:33, 2 November 2017

No idea why iw code was chosen, but I can see that there are no actual message strings for that translation file, so I just removed that one. I fixed the login button as well.

On the way we're doing language selection: We start out by using the language requested by the browser. We record a detected language for authenticated users on first login. Users may manually set their language on their preferences page as well. We use the recorded value for interaction not tied to a browser session (emails). We're relying heavily on Django's i18n features for all of this to just work behind the scenes. Integrating the jquery library might be a difficult fit for this project, as our community expects to be able to use the site without javascript. Having said that, this looks much, much more usable than having a selection list with a ton of items in it. The current ui works for 3 languages, but I could see it getting user-hostile with 100. I'll see how feasible it is to use this when we have javascript, and use the plain form when we don't.

Jsn.sherman (talk)11:29, 3 November 2017

Very belated update: I've reworked the way we're setting the available languages so that Django is checking for the translation directories on startup. We restart the app every time we pull from the repo, so that should make for a nice automated workflow for adding new translations. Django doesn't support all of the languages listed at https://github.com/wikimedia/language-data (although we're definitely in the triple digits), so I added code to use the intersection of Django and wikimedia language codes and the autonyms from the wikimedia language data instead of English language names. I've significantly updated our database configuration so that the content that lives in the db can be localized across the entire intersection of languages without the app falling over from db column and index limitations.

Jsn.sherman (talk)14:03, 2 July 2018

What does happen when Django doesn't support a language? Do we need to add a blacklist/whitelist on our side?

The rest sounds good. Should we start checking about enabling this project on translatewiki.net?

Nike (talk)11:12, 3 July 2018

On its own, Django was falling over dead (500) when it encountered an unknown language code. I've configured our app to just ignore unknown languages. We should probably do a whitelist so that volunteers don't work on translations that aren't provided to users. That data structure currently lives as a python dict, so I could have the platform provide it as a JSON, CSV, or something similar. Other than ironing this detail out, I think we're about ready to go.

Jsn.sherman (talk)17:33, 3 July 2018
 

Looks like our actual supported language count is 83. I was doing quick glances before and was apparently including keys and values in the count. Here's the json whitelist which is computed from the live site. https://wikipedialibrary.wmflabs.org/i18n-whitelist

Jsn.sherman (talk)15:49, 10 July 2018