Re: Suggestion to allow macros that don't expand within "string"

[Daniel Goldman]

Hi Eric,

> Sorry, but the historical m4 language does not have an intrinsic notion
> of a double-quoted string - the only quoting it has is what you get via
> `' (or via whatever you pass to changequote); and the way the m4
> language is designed, the start and end quote work best when they are
> distinct from each other.  Adding in support for quoted tokens whose
> beginning and end token are the same byte is a MAJOR undertaking that
> drastically changes the language; you would no longer be writing m4, but
> a new macro language.  It's not to say it's impossible, but you'd
> definitely be treading in new territory.
>

I already knew the m4 language did not have the intrinsic notion of a 
double-quoted string. I never suggested it did. And I'm well aware that 
`quote delimiters' work best when they are different. I would never 
suggest setting quote delimiters (with m4_changequote) to "", it's a 
very bad idea. So I was a little confused by the emphasis on explaining 
those basic concepts, and wondered if maybe you somewhat misunderstood 
my post...

Just to be clear, my point was that most computer languages, and 
specifically C, DO have the notion of a double-quoted string. And m4 can 
be used to process source code for those languages. So my thinking is m4 
would be more useful if had the option to take double-quoted strings 
into account independent of the quote delimiter (not expand certain 
macros within double-quotes, regardless of the quote delimiter).

Here is a fuller example:

======== Current m4 behavior:

$ cat dq-01.m4
m4_define(`MACRO_1', XXX)m4_dnl
MACRO_1 "MACRO_1"

$ m4 -P dq-01.m4
XXX "XXX"

======== Current cpp behavior:

$ cat dq-01.cpp
#define MACRO_1 XXX
MACRO_1 "MACRO_1"

$ cpp -P dq-01.cpp
XXX "MACRO_1"

======== Possible new *** optional *** m4 behavior:

$ cat dq-02.m4
m4_define2(`MACRO_1', XXX)m4_dnl
MACRO_1 "MACRO_1"

$ m4 -P dq-02.m4
XXX "MACRO_1"

I am not sure that adding support for this kind of option is a "MAJOR 
undertaking", or "drastically changes the language", or creates a "new 
macro language". To me, it seems more a special variant, and I provided 
an outline for logic to add in the support. Of course, I don't really 
know. There might be other factors. It's usually easier to suggest 
changes than to do the detailed coding required. :)

Anyway, in the meantime, I did the work-around, as I suggested in my 
initial post in this thread, to get the behavior I needed for my special 
situation. But that does not address the more general case of macro 
names occurring within "double quote" strings.

> It's open source - you're welcome to try your hand at the improvement.
> But it is such a major undertaking that I'm not holding my breath for it
> from anyone soon.  I don't think you'd even need a new m4_define2;
> rather, just the addition of a new syntactic element for double-quoted
> string and the ability to turn on or off whether double-quoted strings
> are lexed differently.
>
> If you want to try your hand at it, the first thing I'd suggest is
> modifying enum token_type in m4.h to add a new TOKEN_DSTRING, teach
> input.c how to parse double-quoted strings, and figure out what new
> builtin macro to add to be able to toggle whether double-quoted strings
> should be treated specially (and even what their delimiter is) from that
> point on in the processing.
>

I see your suggestion is maybe a macro to change double-quote behavior 
for ALL macros, from that point on in the processing. In my particular 
case, the m4_define2 was what I needed, but I can see how the other "ALL 
macros" behavior might also be useful, maybe even more useful.

I appreciate your coding suggestions, but I'm reluctant to get into 
modifying the source code, even if my personal situation allowed it. If 
the master branch and stable branch need to be reconciled, as suggested 
in the June 5 post, that seems a much higher priority, and in the 
meantime maybe not a good idea for some new coder to be making changes. 
And it seems better (to me) to have an agreement about what to change 
before some new coder starts submitting patches. My guess is that 
everyone might agree that allowing comments to be totally ignored would 
have a higher priority. And obviously there are myriad details in the 
source code that would need to be understood, a big learning curve.

Anyway, I appreciate the work you do. I think it's also helpful to get 
outside comments and suggestions, and that's what I've tried to do. I 
like m4, and have found it useful, but can also see things that seem in 
need of improvement. Maybe eventually these might be done (or judged not 
worth doing).

Thanks,
Daniel

On 6/18/2014 2:45 PM, Eric Blake wrote:
> On 06/18/2014 03:10 PM, Daniel Goldman wrote:
> > For some future m4 version, I'd like to suggest an alternative
> > "m4_define2" (or other more clever name) that is the same as m4_define,
> > but the newly defined macro does not expand if within a double-quoted
> > "string". This would be useful for me, and I think for others too.
>
> Sorry, but the historical m4 language does not have an intrinsic notion
> of a double-quoted string - the only quoting it has is what you get via
> `' (or via whatever you pass to changequote); and the way the m4
> language is designed, the start and end quote work best when they are
> distinct from each other.  Adding in support for quoted tokens whose
> beginning and end token are the same byte is a MAJOR undertaking that
> drastically changes the language; you would no longer be writing m4, but
> a new macro language.  It's not to say it's impossible, but you'd
> definitely be treading in new territory.
>
> > As a workaround, the input file can (possibly) be modified to produce
> > the needed results. It seems the diagnostic macros I use might need to
> > be modified to add another argument with double quote delimiters. In
> > other words, DIAG_INT(MACRO_1) might need to be changed to something
> > like DIAG_INT(MACRO_1, ``MACRO_1''). But this is tedious, and puts the
> > effort on the user. And I already tried it. It's not trivial getting it
> > to work correctly. Instead of making the user flail around, I think it's
> > better to improve m4.
>
> It's open source - you're welcome to try your hand at the improvement.
> But it is such a major undertaking that I'm not holding my breath for it
> from anyone soon.  I don't think you'd even need a new m4_define2;
> rather, just the addition of a new syntactic element for double-quoted
> string and the ability to turn on or off whether double-quoted strings
> are lexed differently.
>
> If you want to try your hand at it, the first thing I'd suggest is
> modifying enum token_type in m4.h to add a new TOKEN_DSTRING, teach
> input.c how to parse double-quoted strings, and figure out what new
> builtin macro to add to be able to toggle whether double-quoted strings
> should be treated specially (and even what their delimiter is) from that
> point on in the processing.