Re: flex beta 2.5.23 released

[Bruce Lilly]

John Millaway wrote:

> <sys/types.h>  has been standard since POSIX.1

POSIX.1 is an operating system standard, not the C language standard.
Note also that POSIX.1 also standardized <unistd.h>, which has been a
problem for some (though at least in that case, configure provides a
HAVE_UNISTD_H macro).  POSIX.1 uses <sys/types.h> for operating system-
specific types, e.g. dev_t, pid_t, ino_t, and uid_t.

POSIX.1 won't solve the problem for non-POSIX systems, obviously. And
there's no mention of int32_t etc. in POSIX.1, at least not in the
1990 version.

> > Moreover, there's no convenient way to define FLEX_NEEDS...... when
> > building flex; configure doesn't do it..
>
>
> Yes it does:
>
>         ./configure CPPFLAGS=-DFLEX_NEEDS_INTEGRAL_TYPES

That's not convenient; that means:
./configure
make
*** error ***
"aw, crap, not again"
./configure "CPPFLAGS=-DFLEX_NEEDS_INTEGRAL_TYPES"
make
*** different error ***
"now what?"

As distributed, flex 2.5.23 doesn't work w/o the macro (because
the appropriate header is not included) or with the macro (because
of a syntax error in the path taken by the preprocessor when the
macro is defined).  So after all of the contortions, flex still
doesn't build as distributed on some ANSI- and POSIX-compliant
systems.

> > And there's no mechanism to provide for it in cases where the
> > generated .c file is compiled on a different system from the one
> > that ran flex to produce that .c file.
>
>
> Flex should generate the same .c file regardless of platform. If you
> found a case where it doesn't, then that's a bug.

I think you misunderstand the issue; the generated .c file contains
no mechanism to choose the correct code at compile-time, which requires
one or more of the following:
1. manual editing of the generated .c file file for each target platform
   (then the .c files are no longer the same...)
2. a scripted version of #1 (same objection)
3. compilation system changes (introduces problems for the sake of
   handling inherently non-portable constructs, may invalidate
   warranty and/or support agreements, may break other programs,
   may be forbidden by local administrative policy)
4. editing of Makefiles, scripts, etc. for each target platform (the
   code is still different for each platform because of the macro's
   effect via the preprocessor, extra work for each target system).
   More important, this doesn't work as there exist standards-compliant
   systems for which the .c files cannot be compiled either with or
   without the macro defined.

The problem is that the generated .c file will work on some systems
and cannot be compiled on other ANSI- and POSIX-compliant systems
regardless of whether or not the FLEX_NEEDS... macro is defined.

There is also an objection in principle: requiring an extremely long
macro to paper over the issue tends to result in atrocities like
cc -DSOME_EXTREMELY_LONG_MACRO_FOR_FLEX 
-DSOME_EXTREMELY_LONG_MACRO_FOR_PROGRAM_X 
-DSOME_EXTREMELY_LONG_MACRO_FOR_PROGRAM_Y 
-DSOME_EXTREMELY_LONG_MACRO_FOR_PROGRAM_Z -o foo a.c b.c c.c d.c

Moreover, you haven't solved the problem; you've magnified it. Instead
of resolving the issue by generating a portable .c file, the non-portability
is extended to every system on which the resulting .c file might be
compiled.

The current stable release of flex, 2.5.4a, does not have this problem. It
generates .c files which are quite portable.

> > Also note that on systems *with* a working <inttypes.h>,
> > that header should be used rather than the definitions in the
> > FLEX_NEEDS........ branch, as the latter may conflict with <inttypes.h>,
> > and <inttypes.h> might be included for use with user-defined functions
> > in the last section of the .l file.
>
>
> The problem is identifying whether or not the given <inttypes.h> is a
> *working* one. It seems that some systems have the file, but the
> contents violate the standard.

The *root* problem is that the intN_t types are inherently non-portable:
* they are optional, even in C99, which means that there are ANSI-compliant
  systems that don't define them
* they *may* be defined (in <stdint.h>, included via <inttypes.h> per C99),
  in which case there may be a conflict with alternate attempts to
  define the same type
* there are similar issues with "long long" and "long long" vs. "long long int",
  so even in the absence of a system-supplied type definition, there is no
  portable way to get an int64_t as a basic type (as opposed to a structure).
  [though it's unlikely that any practical lexical analyzer would require
  tables so large]