Re: Wondering why m4 ignored namespace concept

[Raphael 'kena' Poss]

Dear Daniel,

I have been reading your e-mails and Eric's answers with great interest, and I 
would like to answer a few points (below).

However before I start please let me ask you a question: what *goal* (or goals) 
are you trying to achieve with this conversation?  Are you carrying this 
conversation so that you personally can learn the GNU M4 language and its 
history better? Are you carrying this conversation in order to influence the 
design of future versions of GNU M4? Are you carrying this conversation in 
order to learn how you can personally make changes to the GNU M4 source base? 

Depending on your goals we could certainly "optimize" the conversation and help 
you reach those goals more quickly and with less effort from all parties 
interested.

As to the small points I would like to contribute to:

Op 17 jul 2014, om 11:38 heeft Daniel Goldman het volgende geschreven:

>> Alas, history is not on our side. M4 was defined in 1977,
>> http://dl.acm.org/citation.cfm?doid=367177.367223 [...]
> 
> I'm not sure what the acm citation is, or how it relates.

I have sent you a download link privately in a separate e-mail. By reading it 
you will learn about the original motivations for M4 and its design rationale.


> If you don't see the kludginess of renaming the default names, I don't know 
> what to say.
> What would you think if you saw C code where someone renamed a bunch of C 
> functions? Gawd!!!

For your information, M4 is not the only case where this happens. This is 
precisely what the vast majority of C++ standard libraries do with the 
functions from the C standard library: the C functions in the global namespaces 
are renamed, one by one, so that they come into the "std" namespace (eg. cos -> 
std::cos).

The ability to rename M4 symbols within the language was designed for different 
reasons, and it just so happens this feature can be advantageously exploited to 
solve the "namespace problem" that you find so important. I believe myself it 
is actually an advantage that we can leverage an *existing* feature of M4 to 
solve this problem, without the need for additional development. 

I would even say, the ability to leverage an existing feature of a system to 
solve a problem that was not initially envisioned is a very positive aspect in 
general! The general process of repurposing/recycling some aspects of a system 
to fix some others when deemed necessary is really a quite basic human 
activity, that happens all the time even outside of computing. I honestly fail 
to see what is wrong with that.

> The desire to rename various builtin macros to begin with m4_ prefix makes my 
> point the language would have been better designed to name them that way in 
> the first place. [...]

There is something that Erik did not explain earlier but I believe is relevant 
here: when talking about "historical reasons" we ought to put ourselves in the 
shoes of our forefathers. 

The real question I see you asking is: "Why is it that the initial implementers 
of M4 did not think of putting the prefix "m4_" in front of the name M4 
pre-defined macros?"

There are, I believe, two reasons, maybe equally important:

- they did not know better. The concept of namespace conflicts was not very 
well-known in the 1960's; this was still a time where giving a symbolic name to 
a program fragment was sufficiently new that experts did not yet envision how 
names from different origins may end up colliding down the road. I'd say next 
to M4, most programming languages from that era until the late 1990 never 
considered namespacing until later in their history -- Fortran, Modula, COBOL, 
C, Pascal, BASIC, all of them were designed with many other priorities in mind 
than finding a sound and future-proof naming scheme for predefined language 
building blocks.

- memory was scarce. Seriously, when you have to count every word in your 
program so it can fit in the mere 1K of core memory you have, introducing a 
mandatory prefix of 3 characters to every "important" identifier (which may end 
up occurring many times in a program) forms a non-trivial overhead.


> About the modules, I would suggest maybe we are up to seven wrongs, and that 
> this is perhaps the worst wrong... If all builtins started with m4_ prefix, 
> would you still need to use modules?

Yes, modules are desirable in general, so that the functionality of an M4 
interpreter can be extended/customized by users without requiring a custom 
build/install of the entire M4 package.

> On a practical level, wouldn't the "risk of breaking existing behavior" be 
> removed if m4_ namespace was used?

There is little risk to break existing behavior if the m4_ namespace was used 
"by default", and this is regardless of the existence of modules. One could 
readily imagine an implementation of M4 where all predefined macros are in the 
m4_ namespace by default, with *aliases* also put in the "global namespace" (eg 
`m4_define' aliased to `define') unless -P is defined.

But then, let me ask you: what would we achieve by doing this? (see my 
questions about goals at the beginning)

> The idea of "modules" sounds complex.

Modules are not complex. It may be that you don't know yet what they entail and 
why they are important, but I can certainly confirm (as a long-time user of M4) 
that I would like to see modules become a reality, and they are certainly not 
complex to use nor comprehend.

> And given the stuck nature of development, it would seem better to minimize 
> wasted effort (maybe something we can agree on). We all know it's easy to 
> head down a wrong path, especially with this kind of "freeform" development.

Modules are not a wrong path, in that multiple users have expressed their 
desire/interest in modules in the past.

This is (by the way) a general feature of well-maintained open source projects: 
projects to implement features are always properly discussed and motivated 
before they happen. Rest assured that modules are not a case of a feature in 
need of a problem to solve.

However it is very much so that the motivation for modules has nothing to do 
with the question of whether M4 pre-defined macros should have a m4_ prefix in 
their name or not.

> BTW, why do you say "was to allow modules" (not "is")? Is modules idea thrown 
> out? Is there a list of other ideas for "eventual m4 2.0"?

You can scan the mailing list archives to figure them out. Most of them have 
been discussed on this mailing list already.

>> Here, __line__ was a GNU extension copying after the preprocessor[...]

> I understand why __line__ was named such, and it's all understandable given 
> the lack of a namespace. It just seems obviously kludgy to me, and probably 
> to many others.

This is simply not true. The general idea to use the same name for the same 
thing in different languages is extremely good, because it lowers the cognitive 
overhead for users to keep multiple languages in their head. 

> You misread my question, or I did not state clearly enough. I asked why m4 
> did not use a namespace, when "originally written". Anybody care to answer my 
> question?

See above. "Did not know better" (also: other priorities in the design) and 
"Too little memory".

> But it seems better to acknowledge reality of various flaws, and perhaps 
> eventually fix them. There ARE repercussions, as I mentioned in my post.

Look this is really the point where you should be honest with us (the other 
readers of this mailing list) and state what your motive is.

For your information, everyone with a few years experience developing software 
has internalized the idea that "if it ain't broke, don't fix it". This is a 
mathematical equation that every programmer has in their head: how much benefit 
is obtained by changing something vs. not changing it, and what are the costs 
associated.

In this entire discussion it is pretty clear that the namespacing choices in 
GNU M4 do not currently warrant a change. You have not demonstrated 
convincingly that users of GNU M4 are measurably hindered by the current state 
of affairs, therefore there is no reason yet to do anything about it.

Of course if you just want to understand why things are the way they are, all 
is fine and I am ready to bring knowledge to you (and so is Erik, probably).

But if you intend to change M4, just having philosophical and historical 
conversations will not help. Either you bring objective measures of how the 
current implementation is hindering the progress of programmers; or you 
implement the change yourself and submit a patch to this mailing list; or you 
pay someone a reasonable amount of money to do the change themselves in a 
branch/fork of the repository and submit the patch later from there for further 
discussion. This is open source software, after all.

Best regards,

-- 
Raphael 'kena' Poss · address@hidden
http://staff.science.uva.nl/~poss/