Please stop revert war

Edited by author.
Last edit: 16:32, 12 February 2018

I've said already that the simple replacement (i.e. deletion) of "$2" to translate these was NOT working: you can type a translation like this, it is recorded but immediately marked as "FUZZY" meaning that what you do has the same effect as deleting a translation completely.

The top banner indicates we want to have these translations including adapting the URL. So that's what you want to, but your way does not work at all and will never work.

But "$2" has to be kept somewhere (at least in a comment or otherwise made visible): the translate tool ensures that all original placeholders are present, and if any one is missing, the translation is marked as incomplete/fuzzy and not used at all: the effect is that people will effectively see the untranslated English text as a fallback.

Since the begining this is what I want to avoid. For now what you do is exactly the same has saying that these resources are completely untranslatable and will remain in English for all !

Please suggest something that allows NOT including "$2" without marking the transaltion as "fuzzy" because it is missing : this does not exist for now in the translate tool to mark a variable placeholder as "optional", or explicitly replaced by another content than what is defined in the "var": for now the only existing way is to place the $2 in comments or make it visible.

I don't know why the Upload wizard complains about the presence of HTML comments in resources, when everything else in MediaWiki works like this. May be if HTML comments does not work, the wizard will accept "" with an empty string result, but I doubt it will work if the Upload Wizard in facts performs its own parsing of the resource and expects a format that the Translation tool is currently unable to deliver anywhere.

So yes I suggest fixing the UploadWizard code so that it will parse the wikicode or standard HTML correctly and will not buildup some incoherent presentation of what it contains.

Note that I have not "reverted" things, but documented what was working and what was not, in order to explain (and demonstrate to you) that your attempt was failing all the time.

Initially I placed the HTML comment just after the "[" before the URL, someone said it was parsed by the UploadWizard incorrectly, then I tried to place the $2 AFTER the link (completely outside of it). Waht the UploadWizard does is some internal tricky magic transform, but you've not proposed anything that allows the Translate tool to accept any one of the translations you submitted: in all cases we saw "incomplete translation" statistitics, and many users have tried something; only what I propose is for now aceptable for the translate tool.

We absolutely need a solution to fix either (or both):

  • The UploadWizard (so that it correctly handles the HTML or wiki content and does not reencode it in a non-standard custom way);
  • The translation tool (to mark a variable placeholder where a translation wants to explicitly replace its content without marking the translation as "fuzzy" because it is missing);

otherwise we will never see the correct translations expected (not even yours !).


Note that for URLs to Commons licences all the replacement URLs have the property of sharing the same prefix (and it is the content of $2) but we cannot "glue" "deed.fr" just after $2 because of the current supported syntax of placeholders; if we use "$2deed.fr" the Translation tool will think we want to use a non-existing variable named "2deed" (so the transaltions will be also marked as "fuzzy" and the translation package will remain "incomplete", not 100% translated) :

This is also a defect of the Translation tool which offers no clean way to indicate explicitly where a "$variable-name" is terminated in the translated resource. I think it should support "${2}deed.fr" here (like in Bash command files where this extended syntax is used for that purpose to avoid the tricky escaping) or more generally ${variable-name}, or the alternative syntaxe "%2%" (like in DOS/Windows command files) which is probably simpler to type (the set of characters allowed in variable names between the "%"-pair should be only letters, digits, or some only punctuations: underscore, hyphen, colon, excluding controls and whitespaces, semicolons, commas, full stops, percent, question marks, exclamation marks, single and double quotation marks, and probably others, otherwise there will be "false positive" detection of dummy variables in translated items).

My preference would be the syntax using paired punctuations with ${variable name}, but other suggestions are possible: $<variable name> or $[variable name] or $(variable name) or $"variable name" or $'variable name' (this solution is more liberal on the set of characters we accept in variable names used in translatables source).

Verdy p (talk)15:59, 12 February 2018

Placing a html comment anywhere in the string does not work. That is worse than having the link point to the English version (which has a language selector).

If you want to deviate from the excepted translation (link pointing to the English), you must file a bug or support requests. Suggesting hacks that do not work is not acceptable.

We also export fuzzy translations, so your assumption that fuzzy translations are not exported is not correct.

As a site admin, it is my duty to ensure our translators get good advice, it's not my duty to fix i18n issues in the products.

I'll try to make this very clear: altering the message documentation or translations to work around i18n issue is NOT OKAY. The proper process is to report i18n issues to the developers so that they can make a proper solution.

Nike (talk)16:31, 12 February 2018

"fuzzy" translations will be edited all the time by any translator. You should not use them as a trick in the Upload Wizard. It is really a bad solution which will constantly drive translators to try fixing them.

So what you're saying to translators is to use a trick to insert fuzzy translations that won't be checked at all. No one on TranslateWiki (except you) expect these "fuzzy" strings to be effectively correct. But as you manage this wiki, you should be interested in fixing the hosted Translation tool to support more explicit variable name termination, or marking a required variable as "used" with an empty replacement value which will be absent from the export (but only if the project to translate accepts such absence by explicitly marking a placeholder as optional: this is not the case for many projects that expect ALL placeholders to be present, notably placeholders for C/C++ format strings, an absence of the placeholder causing severe software bugs such as null pointer exceptions, or using integer values in other placeholders as if they were pointer values, causing security breaches such as making local variables hidden on the application stack to become visible).

Please fix the Upload Wizard (it should not use any fuzzy translation at all). Only this Wizard does not follow the MediaWiki wellknown practices and uses custom parsing deviating from recommendations. HTML comments are not "tricks" they are part of the Wiki standard since ever ! but using "fuzzy" translations IS A REAL TRICK which should never happen with TranslateWiki (which should even better not export them at all, they are fuzzy for various reasons, including incorrect syntax, missing required placeholders, unmatched paired punctuation, incorrect encoding,... !)

And the proper way to report i18n issues to developers is via the editable /qqq documentation that they will get from TranslateWiki exports along with translated items for all languages ! This is used in many projects to report cases where there should be plural forms, or genitive forms, or gender variants. Other ways to report these issues are nearly impossible to find in the interface and varies from project to project (there's no standard way to indicate where to submit these issues).

Verdy p (talk)16:37, 12 February 2018
Edited by author.
Last edit: 15:36, 9 July 2019

The real truth is that the new UploadWizard is incorrectly parsinf the "[$2 legal text]" which is supposed to be an external link in the Mediawiki syntax, as if it was a wikilink (normally with the syntax "legal text") and then uses the first parameter of the link (the URL "//creativecommons.org/licenses/.../") by prepending it directly to the base URL of the wiki pages). It is already broken including in English as the final URL (in the generated HTML) is also invalid as it gives (for example on Common) the following HTML code:

  <a href="https://commons.wiki.org/wiki///creativecommons.org/licenses/.../">

For doing this, it makes a special simplified parsing of the translated item. Then it checks the "weakly parsed URL": if it starts by "//" or "http://" or "https://", it won't prepend the base URL of the wiki, but will use that URL directly as is (once again without looking if there are HTML comments or even if in fact its syntax is invalid); if the "weakly parsed URL" starts by anything else, it assumes this is a a FULLPAGENAME, and so it will prepend the wiki's base URL for articles.

Only then it will start converting this broken code using MediaWiki parser to generate the HTML and finally the link.

So the presence of "" at start of the url using the syntax "[url text]" effectively does not work (this was the initial report, concerning ONLY the new broken UploaWizard deployed on Commons, but not the previous one on all other wikis), but the suggestion to put it outseide the link really works: my suggestion was correct, it fixes it and is compatible with BOTH the legacy UploadWizard and the new one (even if the later is still using a severely broken code), and it does not require any fuzzy translations (that are NOT used at all, I can confirm it, they are replaced by the original untranslated English text!)

The only working translations are those using "[$2 some-translated-text]" (e.g. the translation in Chinese) **without** following the indication to localize the URL (this inidcation was not added by me it was a desire to localize it given by the UploadWizard authors themselves (and of course the origional "[$2 untranslated-English-text]". So Chinese does not follow the top indication given to translators in a bold red text at top, and does not attempt to localize the URL at all.

The bug is not in translations, but in the new UploadWizard implementation (as used on commons but not all wikis which still use the former non-broken UploadWizard), it exists independantly of my suggestions or even if one uses the "fuzzy" translations that have removed "$2" and placed another URL there (in which case the fuzzy translation is NOT used at all by former implementations of the UploadWizard in all other wikis).

The bug is really in that new version of the Wizard, not working at all on Commons, whatever we put or don't put in these translations (the new version works ONLY in English where localisation of the URL is not used, even if it could be there to point to the "deed.en" version instead of using the automatic langague selector in CreativeCommons, which may display another language than English, depending on language settings of the webbrowser used to navigate, which may not be always the user's prefered language if the local webbrowser settings cannot be changed by that user, or may not be implemetned at all on some devices not transmitting any "user language" preferences to the web server, or because these browser settings are cleared by privacy tools installed on the same device as the webbrowser).

My suggestion to put the dummy placeholder "" completely outside the link is correct (yes I tested it, others that pretend it does not work have in fact NOT tested it), and fixes the issue, even before the new broken UploadWizard (insufficiently tested) is fixed and it works with legacy implementations of the wizard on other wikis (i.e. most Wikipedias that allow importing images locally instead of on Commons, and almost all non-Wikimedia wikis).

My suggestion is not a hack as you think, it avoids having to use any fuzzy transaltions that are normally rejected everywhere in MediaWiki (replaced by non-fuzzy fallbacks or the untranslated text).

But yes a better solution would be to be able to instruct the "TranslateLinter" used on this wiki to mark the placeholders that are optional, or that can be safely repeated (i.e. instruct if a variable has 0-n occurences allowed, instead of just 1 by default), or that can be safely reordered (most placeholders in C/C++ such as "%s" or "%d" are not reorderable), or that can be modified (e.g. formatting options in C/C++, e.g. for formatting dates or for currency amounts with a different number of decimals: this kind of linting is complex to instruct, and shoudl not be the first pritority you'll want to work on). With just the "occurences number" linting instruction saying that 0 occurences is allowed, the linter will accept the absence of a placeholder in the translation, and will then not mark it as fuzzy.

Summary: fuzzy translations are ALWAYS undesirable they ALWAYS need to be fixed on this site; but saying to project maintainer to still use them will cause much troubles and much more severe bugs in applications, so your suggestion is a real HACK.

Verdy p (talk)19:03, 12 February 2018

There is way to have Translate not mark translations with unused variables as not fuzzy if this is reported . There is also a possibility that this issue will be fixed in UploadWizard if it is reported to the developers in Phabricator.

Again, it's not my duty to fix UploadWizard.

Again, qqq is not the way to communicate with developers. That's what the "Ask question" button is for. It will leave a message in this wiki, which can then be relayed to Phabricator, if necessary.

But all my time goes into trying to make you understand this. I am going to revert the changes to the message documentation since you did not do it. I will block you if you will try to add them back again.

Nike (talk)08:48, 13 February 2018

This question is not manage here but on the actual project where this is discussed. If there's a way to mark translations as not fuzzy, you should document it and explain that to project maintainer (how can they instruct your linter). For now there's no problem at all with the solution exposed, the bug reported on Commons only affects Commons where the new UploadWizard is still in testbed and has various other bugs reported. It start appearing only in January with the recent new version deployed there (but not on other wikis). The need to localize the link is there since the beginning in 2011. The workaround using the commented placeholder has been working since long (except since January on Commons with the new version deployed only there and that has other problems).

No I did not make an edit war, because there's been a precision added on where the commented placeholder (still needed) can be placed (not within the link itself, which was working for many months before and stopped working only in the new UploadWizard only in Commons, but after it: until now this placement was not explicitly indicated because it was not necessary): it was a different solution: There's been a change to place the comment after, and really it works as a workaround for the new UploadWizard bug, but still also with the previous version used everywhere else.

Your revert will just cancel the documented requirement and desire to localize this URL which is there since at least 2011 (not added by me, present since the beginning where it was initially part of an autotranslated template that was present on Common since much longer than just 2011) and this has not been removed.

We currently cannot honor both the 2011 requirement and the (fuzzy) absence of the $2 placeholder when instructions already request to replace the URL.

And asking to Wikimedia developers to use fuzzy translations is a really bad idea you had it is a severe trick when this resource has always been (and remains) a resource in MediaWiki syntax where HTML comments are still safe (and documented as a core feature used everywhere in all MediaWiki wikis).

There will be further bug fixes in the new UploadWizard but the workaround for this case is simple and it really works (those that say the opposite have actually not tested it, they just see the current bug when the comment is at start just gkued before the wanted repleced URL.

Verdy p (talk)11:54, 13 February 2018

I have implemented an alternative workaround where a missing parameter will no longer cause these messages to be fuzzy. However, since you disobeyed my instructions, I have given you 24 hour block.

Nike (talk)14:57, 13 February 2018

And you've not followed the actual bug which was finally recognized in the code and just corrected today ! Before that there was a need of workaround (for the previous version that worked since 2014, only the new version deployed in Commons in January was affected by this bug).

There were also bugs (still remaining) in the way the resources on Translate wiki were imported into the new version (by jenkins-bot) directly as JSON strings with incomplete pars ing.

However the new version still requires Javascript and the classic server-based form still uses the MediaWiki parser which correctly strips the HTML comments and still require them.

The confusion was also in the wrong instructions given at top of the message, asking to adapt the URL, something that actually have NEVER worked and will now NEVER work with the new client-side Javascript-based upload wizard.

So all I did was to signal the bug, and find a workaround for the current standard version (working in the legacy version, before the beta that was deployed in Commons in January without having been really tested!). Only Commons was in trouble (because of the early deployment of this beta), not any other wiki !

Still the beta now on Commons does not work correctly, as it now requires Javascript and the legacy code (which was entirely server-based) has been broken ! This new UploadWizard is then not part of standard MediaWiki, it canot be deployed elsewhere.

The "jenkins-bot" tool that import and convert these strings to JSON files on the repository still needs fixing to correctly support both versions (the new "cute" javascript version, and the classic form processed by the server and not requiring any javascript but with less assistance for selecting licences.

Verdy p (talk)23:25, 7 March 2018