In the mails sent after a password reset and a registration confirmation, not all of the texts are replaced by localized versions. The =%_{}% constructs remain unchanged. One Example (in German, don't mind the domain and names, these only exist on my offline testing machine):

From: TWiki Administrator <webmaster@oook-oook.unseen-university.net>
Subject: TWiki Passwort zurücksetzen für Ponder Stibbons
To: PonderStibbons <haj@oook-oook.unseen-university.net>
Date: Wed, 16 Nov 2005 19:38:07 GMT

Hallo Ponder Stibbons



Login Name "pos"
Ihr Passwort wurde in "6070" geändert. 

%_{"Please visit [_1] to change your password to something more memorable for you.", "http://localhost:8005/bin/view/TWiki/ChangePassword?username=pos;oldpassword=6070"}%

Wenn Sie Fragen haben wenden Sie sich bitte an webmaster.

It appears that xgettext doesn't catch the corresponding strings from the source templates, they are missing in TWiki.pot. The original text, here from mailresetpassword.tmpl, reads:

%_{"Please visit [_1] to change your password to something more memorable for you.", "%SCRIPTURL{"view"}%/%TWIKIWEB%/ChangePassword?username=%LOGINNAME%;oldpassword=%PASSWORD%"}%

Common to all the errors is that the replacement strings to be inserted for the [_n] placeholders contains double quotes. Such constructs aren't parsed successfully by lib/TWiki/I18N/Extract.pm.

I'd prepare a fix, but need a direction: Should I:

1. fix the parser or
2. replace all occurrances the inner double quotes by single quotes, as in the following example:

%_{"Please visit [_1] to change your password to something more memorable for you.", "%SCRIPTURL{'view'}%/%TWIKIWEB%/ChangePassword?username=%LOGINNAME%;oldpassword=%PASSWORD%"}%

in both cases, an extra run of xgettext would be needed to catch all those strings.

...maybe it's about time to apply for checkin rights instead of having some poor guy fiddling with my patches? I honestly promise not to commit my pending registration patches

---

Trying to view a non-existing web, and getting the "create this web" oops page has a similar error:

URL: http://develop.twiki.org/~develop/cgi-bin/view/Bugs2/WebChanges

Result: %_{"You can ~[~[[_1]~]~[create a new web~]~] if you have permission. Contact [_2] if you have any questions.", "http://develop.twiki.org/~develop/cgi-bin/view/TWiki/ManagingWebs?newweb=Bugs2;newtopic=WebChanges#CreateNewWeb", "webmaster"}%

For what it's worth I believe check in rights are the right thing for you, Harald

-- TWiki:Main.SteffenPoulsen#17Nov2005

---

Harald, if you can fix the parser I think that would be a better solution. CC

---

Hm, the problem is a bit deeper.

Maybe the best thing to do is to drop the %_{}% syntax and stick to %MAKETEXT{}%, which is handled by the main TWiki parser.

But: is it really "right" to have non-escaped double quotes inside a double-quoted string ??

AT

---

Antonio is right, the problem is deeper. My perlish assumption that single quotes would do is plain wrong (though TWikiVariables suggests it in some places).

These non-escaped double quotes inside a double quoted string (NEDQUIDS) always make my mental parser itch.

It seems that today NEDQUIDS must be used in topics because %MAKETEXT{}% is evaluated inside-out, but quotes must be escaped in templates because in templates language substitution of %TMPL:MAKETEXT{}% happens first.

I have found alternatives to Antonio's suggestion to drop template maketext and permanently live with NEDQUIDS, but they are so ugly that I deleted them from my comment before saving. Dropping %_{}% would at least make documentation simpler and I18N usage consistent. We have NEDQUIDS everywhere when a functionTag is used as an attribute of another functionTag, so this is legacy ugliness. The patch is going to be fat, though....

-- TWiki:Main.HaraldJoerg

An alternative approach might be to define _ as a TWiki tag, synonymous with MAKETEXT (which I think is possible). You can try it by editing TWiki.pm and adding:

        MAKETEXT          => \&_MAKETEXT,
        _                 => \&_MAKETEXT,

That might work.... (though it may require a tweak to the parser)

CC

No offence meant - but I would prefer not to (Bartleby) tweak the parser.

There are three parsers to look at: xgettext is using I18N/Extract.pm for both templates and topics, %_{}% is recognized by Templates.pm and %MAKETEXT{}% by TWiki.pm. The attribute syntax of %_ is differing from that of MAKETEXT, the latter being the "normal" functionTags one.

Antonio's suggestion, if I understand it correctly, amounts to dropping the parser in Template.pm together with its syntax, and adapt that in Extract.pm to behave like that of TWiki.pm (which is tricky enough). This parser may be the ugliest of the three, but it is widely used for all functionTags, not only for MAKETEXT, and it needs NEDQUIDS for nested functionTags. I just found another one in backlinks.tmpl, nesting %SEARCH{"...%SPACEOUT{"..."}%..."}%.

I admit that I am too chickenhearted to tweak either the functionTags parser in TWiki.pm or the attribute parser in Attrs.pm. Both are used by stuff documented in TWikiVariables. Is that all new in Dakar or might NEDQUIDS constructs be present in Cairo topics?

Replacing all occurrences of %_ into their %MAKETEXT equivalents may be tedious, but it seems to be straightforward. Maybe most of the changes can be done automagically by stealing code from Extract.pm, and if so, it doesn't matter if the result is a modified %_ or a %MAKETEXT. Personally, I'd prefer %MAKETEXT because it makes it clearer that there's TWikiVariable magic being used, and I would not like to expose a tag like %_ to the user, but would as well accept to be overruled here. If it breaks, there's still svn revert -R .

-- TWiki:Main.HaraldJoerg

---

I have prepared a patch according to Antonio's outline to drop %_{}% and %TMPL:MAKETEXT{}% syntax.

• All occurrences of %TMPL:MAKETEXT{}% (zero) and %_{}% (quite a few) are converted to %MAKETEXT{}% with their operands converted to the appropriate syntax.
• I have not eliminated the code to parse/replace %_{}% and %TMPL:MAKETEXT{}% from lib/TWiki/Template.pm.
• I have not aliased %_{}% to %MAKETEXT{}% to make sure that any %_{}% I've missed is still parsed with the correct attribute syntax.
• As a final tweak the code to automatically underline access keys has been copied to lib/TWiki.pm. I am not happy with this because this is the only place where lib/TWiki.pm creates an HTML element.
• The need to escape '&' in maketext's replacement strings to take account for the accesskey internationalisation has not been documented. There's no occurrence of '&&' for that purpose in the templates yet.
• Gotcha: Commas in replacement strings are not possible with %MAKETEXT{}%. I've not thoroughly checked for this.

The patch is at http://pweb.uunet.de/oook.m/twiki/item981.diff, has 2461 lines and some 140kB. Tests pass, with the exception of those for which I've raised Item1003 and Item1005. However, I am not so sure how much I18N the tests actually check for.

This looks hairy.... alternative ideas, better patches or feedback on this one would be welcome.

-- TWiki:Main.HaraldJoerg

Harald, thanks for your help on that. I'll check your patch as soon as I can. I've raised this item to Requirement, since the current state breaks so many things.

AT

I've applied Harald's patch, with some extra work on SVN 7663.

An extra fix in SVN 7664.

It would be nice to have other people looking for potential breaks.

AT

Antonio, I promise you I am looking, and have silently fixed at least one (e.g. 1069)

What feedback are you waiting for? Why can't this topic be closed?

CC

Minor correction: I've missed Main/PatternSkinUserViewTemplate when looking for %_{}%.

Index: data/Main/PatternSkinUserViewTemplate.txt
===================================================================
--- data/Main/PatternSkinUserViewTemplate.txt   (Revision 7675)
+++ data/Main/PatternSkinUserViewTemplate.txt   (Arbeitskopie)
@@ -1,24 +1,24 @@
 %TMPL:INCLUDE{"view"}%
 
-%TMPL:DEF{"active_form"}%<span class="patternButton"><a href="%SCRIPTURL{"edit"}%/%WEB%/%TOPIC%?action=form&t=%GMTIME{"$epoch"}%" rel="nofollow" title='%_{"Edit personal data"}%'>%_{"Edit personal data"}%</a></span>%TMPL:END%
+%TMPL:DEF{"active_form"}%<span class="patternButton"><a href="%SCRIPTURL{"edit"}%/%WEB%/%TOPIC%?action=form&amp;t=%GMTIME{"$epoch"}%" rel="nofollow" title='%MAKETEXT{"Edit personal data"}%'>%MAKETEXT{"Edit personal data"}%</a></span>%TMPL:END%
 %TMPL:DEF{"inactive_form"}% %TMPL:END%
 
 
-%TMPL:DEF{"inactive_edit"}%<span class="patternButton"><a href="%SCRIPTURL{"view"}%/%WEB%/%TOPIC%" rel="nofollow" title='%_{"View topic"}%'>%_{"View topic"}%</a></span>%TMPL:P{"sep"}%<span class="patternButton"><strike>%_{"Edit text"}%</strike></span>%TMPL:END%
+%TMPL:DEF{"inactive_edit"}%<span class="patternButton"><a href="%SCRIPTURL{"view"}%/%WEB%/%TOPIC%" rel="nofollow" title='%MAKETEXT{"View topic"}%'>%MAKETEXT{"View topic"}%</a></span>%TMPL:P{"sep"}%<span class="patternButton"><strike>%MAKETEXT{"Edit text"}%</strike></span>%TMPL:END%
 
 
 
-%TMPL:DEF{"edit_topic"}%%_{"&Edit text"}%%TMPL:END%
+%TMPL:DEF{"edit_topic"}%%MAKETEXT{"&Edit text"}%%TMPL:END%
 
 
 
-%TMPL:DEF{"active_edit"}%%TMPL:P{context="inactive" then="inactive_form" else="active_form"}%%TMPL:P{"sep"}%<span class="patternButton"><a href="%SCRIPTURL{"edit"}%/%WEB%/%TOPIC%?action=text&t=%GMTIME{"$epoch"}%" rel="nofollow" title='%_{"Edit this topic text"}%' accesskey="E">%TMPL:P{context="new_topic" then="create_topic" else="edit_topic"}%</a></span>%TMPL:P{context="WysiwygPluginEnabled" then="wysiwyg"}%%TMPL:END%
+%TMPL:DEF{"active_edit"}%%TMPL:P{context="inactive" then="inactive_form" else="active_form"}%%TMPL:P{"sep"}%<span class="patternButton"><a href="%SCRIPTURL{"edit"}%/%WEB%/%TOPIC%?action=text&amp;t=%GMTIME{"$epoch"}%" rel="nofollow" title='%MAKETEXT{"Edit this topic text"}%' accesskey="E">%TMPL:P{context="new_topic" then="create_topic" else="edit_topic"}%</a></span>%TMPL:P{context="WysiwygPluginEnabled" then="wysiwyg"}%%TMPL:END%
 
 %TMPL:DEF{"content"}%%TMPL:P{"form"}%
-<div class="twikiHidden"><hr /></div><a name="Content"> </a><div class="twikiHidden"> %_{"Start of topic"}% | <a href="#Actions">%_{"Skip to actions"}%</a><hr /></div>
+<div class="twikiHidden"><hr /></div><a name="Content"> </a><div class="twikiHidden"> %MAKETEXT{"Start of topic"}% | <a href="#Actions">%MAKETEXT{"Skip to actions"}%</a><hr /></div>
 <div class="patternTopic"><!-- hidden text

--><br class="twikiClear" />
 #TopicEnd
-</div><div class="twikiHidden"><hr />%_{"End of topic"}%<br /><a href="#Actions">%_{"Skip to actions"}%</a> | <a href="#PageTop">%_{"Back to top"}%</a><hr /></div>
+</div><div class="twikiHidden"><hr />%MAKETEXT{"End of topic"}%<br /><a href="#Actions">%MAKETEXT{"Skip to actions"}%</a> | <a href="#PageTop">%MAKETEXT{"Back to top"}%</a><hr /></div>
 %TMPL:P{"attachments"}%%TMPL:P{"topicaction"}%
 %TMPL:P{"topicinfo"}%%TMPL:END%
 

-- TWiki:Main.HaraldJoerg

Applied Harald's last change on SVN 7680.

I'm closing this, since I think we're ok now.

AT