Re: m4-1.4.x fails to build git Autoconf for some x (was: branch-1_4 cannot build Autoconf 2.59 any more)

[Keith Marshall]

On Monday 24 March 2008 20:28, Ralf Wildenhues wrote:
> > The problem arises because the vanilla build with MinGW leaves the
> > stdio streams in O_TEXT mode, resulting in CRLF output, whereas the
> > MSYS build uses O_BINARY mode for these streams, resulting in just
> > LF output; the residual CRs from O_TEXT output seem to confuse the
> > autoconf build process, in the freezing stage.
>
> It's not clear to me from this description whether it is vanilla m4
> that is broken, or the Autoconf freeze code (and it's just the
> workaround that is in m4).  IOW: what would be The Right Thing to do?

That's a tricky one!  Maybe a truly native m4 for Win32 really does need 
to be O_TEXT; after all that is the native standard for the platform.

However, MSYS aims to provide a bridge between POSIXy build systems and 
the native Win32 environment; MSYS is strictly O_BINARY, and problems 
do seem to arise, when O_TEXT tools are mixed into the chain.

Microsoft themselves contribute to the confusion, because their C/C++ 
runtime translates CRLF to LF on input of an O_TEXT stream, and 
conversely, translates LF to CRLF on output to such a stream.  Thus 
MinGW, which is built entirely natively, with O_TEXT for source code 
streams, sees only apparently LF streams when reading source files, 
regardless of the actual line termination style of the file.

OTOH, because MSYS is strictly O_BINARY, this I/O translation does not 
occur, so if an MSYS application reads CRLF input, it gets to see CRLF 
and must be prepared to handle it appropriately.  When you run autoconf 
under MSYS, you use the MSYS O_BINARY perl, and if you add an O_TEXT m4 
into the chain, the CRLF line terminations are not correctly handled, 
with the CR being interpreted as a component of the data, not of the 
line terminator.

Usually, when I'm faced with such issues, I force the input stream to be 
O_BINARY, then explicitly handle the line terminations in my own code, 
discarding any single CR immediately followed by LF; maybe autoconf 
needs to do likewise, in the freezing code, if there is a desire to 
support an O_TEXT m4 in conjunction with MSYS.

Regards,
Keith.