Re: Wondering why m4 ignored namespace concept

[Daniel Goldman]

On 7/17/2014 3:21 AM, Raphael 'kena' Poss wrote:
> Dear Daniel,
>
> I have been reading your e-mails and Eric's answers with great interest, and I 
> would like to answer a few points (below).
>
> However before I start please let me ask you a question: what *goal* (or goals) 
> are you trying to achieve with this conversation?  Are you carrying this 
> conversation so that you personally can learn the GNU M4 language and its 
> history better? Are you carrying this conversation in order to influence the 
> design of future versions of GNU M4? Are you carrying this conversation in 
> order to learn how you can personally make changes to the GNU M4 source base?

I think my original post was pretty clear. Here is what I said:

1) Find out the reasoning (if any) why the original decision was made
not to always preface builtins with m4_ prefix.

2) Point out something that seems (to me) obviously inferior, but which
nobody else seems to have noted.

3) If my comments are valid, suggest that in some "better m4 in distant
future", m4 always use m4_ prefix for builtins.

In case still not clear, for this particular post, the purpose was 
"influence the design of future versions of GNU M4".

> > I'm not sure what the acm citation is, or how it relates.
>
> I have sent you a download link privately in a separate e-mail. By reading it 
> you will learn about the original motivations for M4 and its design rationale.

Thanks for sending. I had seen that article before. Unfortunately, it 
does not shed any light on my question about namespaces, so I don't see 
the point. But thanks for clearing up what Eric was trying to send.

> > If you don't see the kludginess of renaming the default names, I don't know 
> > what to say.
> > What would you think if you saw C code where someone renamed a bunch of C 
> > functions? Gawd!!!
>
> For your information, M4 is not the only case where this happens. This is precisely what the 
> vast majority of C++ standard libraries do with the functions from the C standard library: 
> the C functions in the global namespaces are renamed, one by one, so that they come into the 
> "std" namespace (eg. cos -> std::cos).

The coding horror I described is not "precisely" what the C++ standard 
libraries do. It's not even close, you are way off on a tangent. My post 
was clear, but here I'll spell out in detail the analogous totally 
undesirable coding practice I was referring to:

#define c_strncat strncat
#define c_strncpy strncpy
etc.

If you don't see the kludginess of that, compared with just using the 
given C function names, I don't know what to say. I think you are trying 
to defend the indefensible.

> The ability to rename M4 symbols within the language was designed for different reasons, 
> and it just so happens this feature can be advantageously exploited to solve the 
> "namespace problem" that you find so important. I believe myself it is actually 
> an advantage that we can leverage an *existing* feature of M4 to solve this problem, 
> without the need for additional development.
>
> I would even say, the ability to leverage an existing feature of a system to 
> solve a problem that was not initially envisioned is a very positive aspect in 
> general! The general process of repurposing/recycling some aspects of a system 
> to fix some others when deemed necessary is really a quite basic human 
> activity, that happens all the time even outside of computing. I honestly fail 
> to see what is wrong with that.

I'm positive about m4. But that doesn't mean ignoring obvious failings. 
We all use workarounds and kludges, but those stopgaps are not 
desirable, it would be better if they were not needed in the first 
place. You put "namespace problem" in quotes, implying it is not a real 
problem, just some petty preference of mine. I would take "namespace 
problem" out of quotes. In this context, I think it's a real problem, 
because m4 uses arbitrary text input files mixed with m4 builtin commands.

> > The desire to rename various builtin macros to begin with m4_ prefix makes my 
> > point the language would have been better designed to name them that way in the 
> > first place. [...]
>
> There is something that Erik did not explain earlier but I believe is relevant here: when 
> talking about "historical reasons" we ought to put ourselves in the shoes of 
> our forefathers.
>
> The real question I see you asking is: "Why is it that the initial implementers of M4 did not 
> think of putting the prefix "m4_" in front of the name M4 pre-defined macros?"
>
> There are, I believe, two reasons, maybe equally important:
>
> - they did not know better. The concept of namespace conflicts was not very 
> well-known in the 1960's; this was still a time where giving a symbolic name to 
> a program fragment was sufficiently new that experts did not yet envision how 
> names from different origins may end up colliding down the road. I'd say next 
> to M4, most programming languages from that era until the late 1990 never 
> considered namespacing until later in their history -- Fortran, Modula, COBOL, 
> C, Pascal, BASIC, all of them were designed with many other priorities in mind 
> than finding a sound and future-proof naming scheme for predefined language 
> building blocks.

That was my guess, that they did not know better. Just wanted to make a 
fact-based (and I think important) observation about m4 that nobody else 
seems to have ever brought up before.

> - memory was scarce. Seriously, when you have to count every word in your program so it 
> can fit in the mere 1K of core memory you have, introducing a mandatory prefix of 3 
> characters to every "important" identifier (which may end up occurring many 
> times in a program) forms a non-trivial overhead.

Yes, that is a possibility, and UNIX does have a history of being terse. 
But there is no statement or analysis to that effect that they thought 
of using an m4_ prefix and decided not to because of memory constraints. 
And it would have to be demonstrated that leaving off the m4_ would be a 
"non-trivial overhead". The more likely reason is the previous reason, 
it just never crossed their mind. We will never know.

BTW, this totally does not apply to C or the other languages you 
mentioned above, and in the email you sent, which I appreciate your 
asking around about. I have no problem with the C naming convention that 
was originally adapted. It's a totally different situation, C and m4. I 
don't understand why you mentioned C.

> Modules are not complex. It may be that you don't know yet what they entail and 
> why they are important, but I can certainly confirm (as a long-time user of M4) 
> that I would like to see modules become a reality, and they are certainly not 
> complex to use nor comprehend.

I think we are off topic here, talking about modules for next version of 
m4. I didn't bring up about modules, Eric mentioned it. They seem 
undesirable to me in this context, on the face of it, in terms of adding 
complexity to the user and the programmer. Just in case there is any 
question, there is a huge distinction between "software modules" and 
"modular programming". If that is unclear, please make a separate post, 
it's off topic here. Anyway, it just came up, and I offered my opinion. 
I know why modules are important in some contexts, eg apache. I didn't 
see them as being a good idea in the m4 context, but I could certainly 
be wrong.

> > And given the stuck nature of development, it would seem better to minimize wasted effort 
> > (maybe something we can agree on). We all know it's easy to head down a wrong path, 
> > especially with this kind of "freeform" development.
>
> Modules are not a wrong path, in that multiple users have expressed their 
> desire/interest in modules in the past.
>
> This is (by the way) a general feature of well-maintained open source projects: 
> projects to implement features are always properly discussed and motivated 
> before they happen. Rest assured that modules are not a case of a feature in 
> need of a problem to solve.
>
> However it is very much so that the motivation for modules has nothing to do 
> with the question of whether M4 pre-defined macros should have a m4_ prefix in 
> their name or not.

The idea of modules has "nothing to do" with question of m4 builtins 
having m4_ prefix? I find that hard to believe. Anyway, since you 
mentioned "multiple users", I looked to see if any previous discussion 
of modules on this group, out of interest. I could not find much. But 
http://lists.gnu.org/archive/html/m4-discuss/2006-09/msg00002.html seems 
at least to reinforce my position that the lack of namespace is a problem.

> > BTW, why do you say "was to allow modules" (not "is")? Is modules idea thrown out? Is 
> > there a list of other ideas for "eventual m4 2.0"?
>
> You can scan the mailing list archives to figure them out. Most of them have 
> been discussed on this mailing list already.
>
> > > Here, __line__ was a GNU extension copying after the preprocessor[...]
>
> > I understand why __line__ was named such, and it's all understandable given the 
> > lack of a namespace. It just seems obviously kludgy to me, and probably to many 
> > others.
>
> This is simply not true. The general idea to use the same name for the same 
> thing in different languages is extremely good, because it lowers the cognitive 
> overhead for users to keep multiple languages in their head.

How can you say it is "simply not true" that it "seems obviously kludgy 
to me". It seems obviously kludgy to me! Who are you to deny my 
reaction? Do you control my feelings?

Back to reality, as a sample of one, I do not find the cognitive 
overhead of m4_line syntax to be higher than the cognitive overhead of 
__line__ syntax. And I use cpp a lot. And you're missing the point that 
I'm pretty much forced to use m4___line__ syntax. What about the 
cognitive overhead of that syntax?

> > But it seems better to acknowledge reality of various flaws, and perhaps 
> > eventually fix them. There ARE repercussions, as I mentioned in my post.
>
> Look this is really the point where you should be honest with us (the other 
> readers of this mailing list) and state what your motive is.

This business that I should be "honest with us" and "state what your 
motive is" is an insult. It suggests I am dishonest and have some 
unstated motive. It just leads to deterioration of any discussion, and I 
think detracts from the purpose of this discussion group. My words stand 
on their own. I plan to stick with specific questions and suggestions. 
If you have something specific you want to ask, go ahead, but if it's 
this kind of personal sidetrack, it would probably be better done in a 
personal email, not to the group. Assuming there is even anyone else 
listening. (echo, echo)

> For your information, everyone with a few years experience developing software has 
> internalized the idea that "if it ain't broke, don't fix it". This is a 
> mathematical equation that every programmer has in their head: how much benefit is 
> obtained by changing something vs. not changing it, and what are the costs associated.

That's pretty common sense about costs and benefits. Do you think you 
are telling me (or anybody) something I don't know? If anything, "if it 
ain't broke, don't fix it" is over-applied. I know it's a widespread 
practice, at times unavoidable, at times even the best course. But I 
think it is fact=based that the long-term consequences of letting known 
problems fester and persist are bad. Good software gets refactored all 
the time. And BTW, you brought up about "if it ain't broke, don't fix 
it", yet you accuse me of "philosophizing"!

> In this entire discussion it is pretty clear that the namespacing choices in 
> GNU M4 do not currently warrant a change. You have not demonstrated 
> convincingly that users of GNU M4 are measurably hindered by the current state 
> of affairs, therefore there is no reason yet to do anything about it.
>
> Of course if you just want to understand why things are the way they are, all 
> is fine and I am ready to bring knowledge to you (and so is Erik, probably).
>
> But if you intend to change M4, just having philosophical and historical 
> conversations will not help. Either you bring objective measures of how the 
> current implementation is hindering the progress of programmers; or you 
> implement the change yourself and submit a patch to this mailing list; or you 
> pay someone a reasonable amount of money to do the change themselves in a 
> branch/fork of the repository and submit the patch later from there for further 
> discussion. This is open source software, after all.

Your suggestion that I am engaged in "philosophical and historical 
conversations" is inaccurate. And of course the word "philosophical" is 
so vague, it's an easy pejorative to use.

The only reason I asked the historical question was to try to figure out 
the reason (if any) namespaces weren't used. There was no reason that we 
can verify, it was apparently just a bad decision. I am not an "m4 
historian", don't real care about the history except how it relates to 
the present.

I think my questions were of a very practical nature, nothing to do with 
"philosophy". The namespace is what the user sees. That is certainly 
very practical. builtin names can conflict with names in the input file. 
That also seems very practical to me. You say all this has nothing to do 
with the proposed m4 modules. I find that hard to believe.

In this particular case, your suggestion for me to just "submit a patch" 
totally doesn't make sense. It's so easy for you to say, but so out of 
touch with practicality. I'm already doing plenty making this post and 
starting this discussion.

Daniel