Re: [PATCH 5/8] parsers: support translatable token aliases

[Adrian Vogelsgesang]

Hi Akim,

Thanks for adding that functionality.
That should make it easier to extract the translatable changes.

Regarding your previous email: You are right, “parsing” was an overstatement.
We are scanning the code for known patterns, and tracking the values of a few 
variables doing so.
Too tightly coupled to C++ to work with grammar files, but should be 
straightforward to adapt.

The “N_” should make things a bit easier, but I am not sure yet, if I will in 
the end end up parsing
the grammar file instead.

Thanks for your help, anyway! :)
(and sorry for the late reply)

Cheers,
Adrian

From: Akim Demaille <address@hidden>
Date: Saturday, 5 January 2019 at 15:42
To: Akim Demaille <address@hidden>
Cc: Adrian Vogelsgesang <address@hidden>, Rici Lake <address@hidden>, Paul 
Eggert <address@hidden>, "address@hidden" <address@hidden>
Subject: Re: [PATCH 5/8] parsers: support translatable token aliases



> Le 4 janv. 2019 à 08:06, Akim Demaille <address@hidden> a écrit :
> 
> Hi Adrian,
> 
> I don't want to generate yet another file. But we could also leave some 
> markup in the generated file to make it easier for tools that don't want to 
> read the *.y file. Gettext supports N_("foo"), where N_ is
> 
> #define N_(S) S
> 
> so that their gatherer can spot the strings to be translated, but that will 
> be translated later with _. Wouldn't that work for you?

The patch below implements what I meant. In the case of Bison itself, that's 
the resulting change:

> static const char *const yytname[] =
> {
> - "\"end of file\"", "error", "$undefined", "\"string\"", "\"%token\"",
> - "\"%nterm\"", "\"%type\"", "\"%destructor\"", "\"%printer\"",
> - "\"%left\"", "\"%right\"", "\"%nonassoc\"", "\"%precedence\"",
> - "\"%prec\"", "\"%dprec\"", "\"%merge\"", "\"%code\"",
> - "\"%default-prec\"", "\"%define\"", "\"%defines\"", "\"%expect\"",
> - "\"%expect-rr\"", "\"%<flag>\"", "\"%file-prefix\"", "\"%glr-parser\"",
> - "\"%initial-action\"", "\"%language\"", "\"%name-prefix\"",
> - "\"%no-default-prec\"", "\"%no-lines\"", "\"%nondeterministic-parser\"",
> - "\"%output\"", "\"%require\"", "\"%skeleton\"", "\"%start\"",
> - "\"%token-table\"", "\"%verbose\"", "\"%yacc\"", "\"{...}\"",
> - "\"%?{...}\"", "\"[identifier]\"", "\"char\"", "\"epilogue\"", "\"=\"",
> - "\"identifier\"", "\"identifier:\"", "\"%%\"", "\"|\"", "\"%{...%}\"",
> - "\";\"", "\"<tag>\"", "\"<*>\"", "\"<>\"", "\"integer\"", "\"%param\"",
> - "\"%union\"", "\"%empty\"", "$accept", "input", "prologue_declarations",
> + N_("end of file"), "error", "$undefined", N_("string"),
> + N_("translatable string"), "%token", "%nterm", "%type", "%destructor",
> + "%printer", "%left", "%right", "%nonassoc", "%precedence", "%prec",
> + "%dprec", "%merge", "%code", "%default-prec", "%define", "%defines",
> + "%expect", "%expect-rr", "%<flag>", "%file-prefix", "%glr-parser",
> + "%initial-action", "%language", "%name-prefix", "%no-default-prec",
> + "%no-lines", "%nondeterministic-parser", "%output", "%require",
> + "%skeleton", "%start", "%token-table", "%verbose", "%yacc", "{...}",
> + "%?{...}", N_("[identifier]"), N_("character literal"), N_("epilogue"),
> + "=", N_("identifier"), N_("identifier:"), "%%", "|", "%{...%}", ";",
> + N_("<tag>"), "<*>", "<>", N_("integer literal"), "%param", "%union",
> + "%empty", "$accept", "input", "prologue_declarations",


The current state of this proposal is currently in the branch token-i18n, on 
both savannah and address@hidden:akimd/bison.git.



commit 35df9e65e396a2f6e7ed89f20b814aa841f22d08
Author: Akim Demaille <address@hidden>
Date: Sat Jan 5 15:23:28 2019 +0100

parsers: issue tname with i18n markup

Some users would like to avoid having to "parser" the *.y file to find
the strings to translate. Let's issue the translatable tokens with N_
to allow "parsing" the generated parsers instead.

See
https://lists.gnu.org/archive/html/bison-patches/2019-01/msg00015.html

* src/output.c (prepare_symbols): Issue tname with N_() markup.

diff --git a/src/output.c b/src/output.c
index 32e4e5ec..8ef462f9 100644
--- a/src/output.c
+++ b/src/output.c
@@ -175,7 +175,8 @@ prepare_symbols (void)
: quotearg_alloc (symbols[i]->tag, -1, qo);
/* Width of the next token, including the two quotes, the
comma and the space. */
- int width = strlen (cp) + 2;
+ int width
+ = strlen (cp) + 2 + (symbols[i]->translatable ? strlen ("N_()") : 0);

if (j + width > 75)
{
@@ -185,7 +186,11 @@ prepare_symbols (void)

if (i)
obstack_1grow (&format_obstack, ' ');
+ if (symbols[i]->translatable)
+ obstack_sgrow (&format_obstack, "N_(");
obstack_escape (&format_obstack, cp);
+ if (symbols[i]->translatable)
+ obstack_1grow (&format_obstack, ')');
free (cp);
obstack_1grow (&format_obstack, ',');
j += width;
diff --git a/tests/calc.at b/tests/calc.at
index f9919bb6..521f983e 100644
--- a/tests/calc.at
+++ b/tests/calc.at
@@ -278,6 +278,7 @@ static int power (int base, int exponent);
]AT_YYERROR_DECLARE[
]AT_YYLEX_DECLARE_EXTERN[

+#define N_
]AT_TOKEN_TRANSLATE_IF([[
static
const char *