Re: Wondering why m4 ignored namespace concept

[Daniel Goldman]

On 7/15/2014 6:03 AM, Eric Blake wrote:
> On 07/14/2014 11:45 PM, Daniel Goldman wrote:
> > When I started using m4, m4exit and m4wrap seemed pretty geeky,
> > inconsistently named. Since I always use -P option, these get converted
> > to m4_m4exit and m4_m4wrap. Seems pretty kludgy. But I make do. It seems
> > that the odd m4exit and m4wrap names were to protect legacy input files
> > that have the word "exit" or "wrap" in them, as apparently these were
> > builtins added later. Just a guess. If my guess is wrong, the odd names
> > are even odder.
>
> Alas, history is not on our side. M4 was defined in 1977,
> http://dl.acm.org/citation.cfm?doid=367177.367223, which lacked wrap and
> exit builtins.  But as other people started using the language, those
> builtins were soon added by third parties.  By the time the GNU project
> decided to add an implementation of m4, in 1990, we were already stuck
> with the names m4exit and m4wrap in order to match existing practice of
> other implementations; and this practice has actually been codified into
> POSIX:
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/m4.html

I'm not sure what the acm citation is, or how it relates. It says it was 
published in 1960 (?). I'm not going to pay $15 :( to find out more, as 
the link requires. The POSIX text says 2001, I'm not sure how that 
relates to the GNU development. I wouldn't know if GNU m4 was already 
"stuck with the names", or not. I'm not saying you are wrong, I'm not an 
"m4 historian", but the links you cite don't seem to support GNU m4 
being "stuck".

Anyway, I never raised the issue of whether GNU m4 was "stuck with the 
names", and don't really care at this point, as my post is really about 
what "should be". To help figure that out, my post asked why the 
decision was made when m4 was "originally developed", I did not ask 
about GNU m4. If some advantage to the "no namespace" decision, that 
would help me to know. If there was not, then I'm afraid this is another 
example where the GNU m4 manual exaggerates / makes a mis-statement. It 
says the naming convention is an "innovation" (!) and "generally useful" 
(!), implicitly in comparison with always using m4_ prefix for builtins. 
I don't buy that.

And again, my post is not to blame anyone, all those old developers from 
decades ago are perhaps dead anyway at this point (but it would great if 
some of them were lurking on this group, and could spell out why the 
original decision was made). My post is to think about what the right 
method is, even if that can't be changed today. Nothing lasts forever. 
It would be naive to bet that m4 will necessarily be around in 10 years 
(or maybe 25, you get the point).

Anyway, what you say agrees with what I'm saying, even if perhaps you 
didn't want to say so. When m4exit and m4wrap builtins got added, the 
names ended up kludgy, through no fault of the "third parties", but 
because m4 had no namespace concept, as I said in my post.

> > I adapted. I'm used to geeky things, it comes with the territory. m4 has
> > a lot of good points, so I put up with the bad. I always use -P as a
> > workaround, so I am not really "suffering". But the naming system seems
> > bad looking and unfriendly. I would say the language is "suffering". If
> > anyone wonders why m4 is not more popular, here is one reason. I speak
> > from direct experience, as a new user.
>
> You _do_ realize that you are free to rename macros into a consistent
> namespace, right?  For example, the very first thing autoconf does when
> initializing m4, right after using changequote to use [] instead of `',
> is to rename all the builtin macros into a single namespace.  Thus,
> autoconf has 'm4_define' and 'm4_exit', not 'm4_m4exit'.  It's as simple as:
>
> changequote([,])
> define([m4_define], defn([define]))
> m4_define([m4_defn], defn([defn]))
> m4_define([m4_exit], m4_defn([m4exit]))
> m4_define([m4_undefine], m4_defn([undefine]))
> m4_undefine([define])
> m4_undefine([defn])
> m4_undefine([m4exit])
> m4_undefine([undefine])
>
> and so on.

Yes, I knew about renaming. But five wrongs don't make a right. Or maybe 
it's up to six. :) The renaming is a kind of work-around. It's not 
terrible, it's just kind of pitiful.

If you don't see the kludginess of renaming the default names, I don't 
know what to say. What would you think if you saw C code where someone 
renamed a bunch of C functions? Gawd!!! The desire to rename various 
builtin macros to begin with m4_ prefix makes my point the language 
would have been better designed to name them that way in the first 
place. That's all I'm saying. You could just say "you're right" and move 
on. I don't see the point in trying to defend the indefensible. :) On 
the other hand, if there is some advantage to NOT using a namespace, I'm 
all ears.

BTW, any advantage of renaming over using -P option? I don't see any. -P 
seems lot simpler to me as a workaround.

> > "Two wrongs do not make a right." - *** Wrong #2. Starting backwards, I
> > would suggest wrong #2 was to use names like m4exit and m4wrap. But I
> > guess a necessary evil at that point. *** Wrong #1. I would suggest it
> > was a fundamental wrong to use "define", "include", etc. It seems
> > obvious (to me) these should have ALWAYS been m4_define, etc. There is
> > this concept of "namespaces" to prevent naming conflicts. I would hope
> > this might have crossed someone's mind way back when. No normal usage is
> > going to use "m4_define" in an input file other than as a builtin.
> > Obviously, various builtins can easily be used all over the place.
> >
> > So my suggestion is:
> >
> > 1) Go back in a time machine. (That's a joke) :)
>
> No, it's not.  It's the only way we can change this.
>
> > 2) Make all builtins like m4_define, m4_include, etc.
>
> Yes, you are free to do this yourself.  Autoconf does.
>
> > 3) Get rid of -P option (not needed).
>
> We can't drop it now; people have already come to rely on our extension,
> even though it is not defined by POSIX.
>
> > 4) If new builtins added, use m4_exit, m4_wrap, etc.

You missed my point that 1) to 4) ("my suggestion") were all together. 
And I tried to make it extremely obvious that 1) to 4) were intended in 
a humorous light (since we cannot go back in time). Obviously, I am not 
suggesting to drop -P now, -P would only be dropped if m4 had namespace.

> GNU M4 added several builtins that were not existing practice at the
> time, such as 'regex'.  If we had a time machine, then yes, any new
> macros added at that time could have been namespaced, keeping only the
> original portable names as bare.  But as we've already argued, it's so
> trivially easy to rename a macro into whatever namespace you want that
> it isn't worth the extra typing of new builtin names, and instead leave
> it up to the end user to do it.
>
> At this point in time, we are unlikely to add any new builtins by
> default.  One of the ideas for an eventual m4 2.0 was to allow modules,
> where you could add new builtins by loading a module - but the point
> remains that those builtins are not loaded by default, but by explicit
> action; and therefore, there is no risk of breaking existing behavior if
> you do not load the new module, and conversely, anyone loading a module
> still has control to rename the macros into a namespace.

About the modules, I would suggest maybe we are up to seven wrongs, and 
that this is perhaps the worst wrong... If all builtins started with m4_ 
prefix, would you still need to use modules? On a practical level, 
wouldn't the "risk of breaking existing behavior" be removed if m4_ 
namespace was used?

Given the current reality, perhaps modules are a "necessary evil". But I 
would guess it would be easier to program and use m4 in "not modules 
mode" vs "modules mode". The idea of "modules" sounds complex. And given 
the stuck nature of development, it would seem better to minimize wasted 
effort (maybe something we can agree on). We all know it's easy to head 
down a wrong path, especially with this kind of "freeform" development. 
So I think it helps to know the path was wrong (as I think it was with 
ignoring module concept). Maybe eventually we can get on the right path.

BTW, why do you say "was to allow modules" (not "is")? Is modules idea 
thrown out? Is there a list of other ideas for "eventual m4 2.0"?

> > Besides simplifying things, preventing conflicts, and looking a lot
> > better, another advantage is to improve searching and maintenance. The
> > user can easily find all instances of m4_define, "\<m4_[A-Za-z_0-9]*\>"
> > etc., in input files, on this discussion group, etc. I find it useful.
> >
> > My suggestion would also get rid of the geekiness with __line__ ->
> > m4___line__ where I think m4_line would have been a lot better.
>
> Here, __line__ was a GNU extension copying after the preprocessor; but
> consistency argues that -P always adding 'm4_' instead of completely
> renaming the macro is easier to do.  But again, you are free to redefine
> it to whatever name you want.

I understand why __line__ was named such, and it's all understandable 
given the lack of a namespace. It just seems obviously kludgy to me, and 
probably to many others.

> > Here (finally!) is my question: Does anyone know why builtins were not
> > required to start with m4_ prefix? Basically, what is the advantage of
> > not using a namespace?
>
> Historical compatibility.

You misread my question, or I did not state clearly enough. I asked why 
m4 did not use a namespace, when "originally written". Anybody care to 
answer my question? As I said, I don't think there was any advantage.

> > But I'm not really expecting anything to happen. I've already gotten
> > that message pretty clearly! I'm not blaming anyone, it's just the way
> > things are. Perhaps hardly anybody cares, or is even listening. Or
> > perhaps users are satisfied with the current state of affairs, perhaps
> > some even revel in the complexity and obscurity. Anyway, the purpose of
> > this post, if anyone still reading, :) is to:
> >
> > 1) Find out the reasoning (if any) why the original decision was made
> > not to always preface builtins with m4_ prefix.
>
> Because GNU m4 was just copying what others had already done, and then
> adding some more builtins at the time.

Again, my meaning did not come across. I was not asking about GNU m4, 
but about the "original decision" when m4 was developed. I thought I 
made that pretty clear.

> > 2) Point out something that seems (to me) obviously inferior, but which
> > nobody else seems to have noted.
> >
> > 3) If my comments are valid, suggest that in some "better m4 in distant
> > future", m4 always use m4_ prefix for builtins.
>
> Unlikely to happen.  But thanks for your thoughtful analysis.

You're welcome. But apparently you don't agree with the "thoughtful 
analysis". Seems pretty obvious to me that m4 was originally poorly 
designed because did not require m4_ prefix. I'm still going to use m4, 
good, bad, and ugly! But it seems better to acknowledge reality of 
various flaws, and perhaps eventually fix them. There ARE repercussions, 
as I mentioned in my post. And I just learned a new one today, about the 
modules, this namespace business seems to perhaps affect future development.

I'm sure you heard the story about fishing the drowning people out of 
the river? Eventually, someone got the bright idea to fix the bridge...

A lot you didn't respond to, but beggars can't be choosers, I appreciate 
your response. Anybody else have any thoughts? Anybody there? Hello? 
(echo, echo) :)

Daniel