Suggestion to add "m4 --flush-output" option

[Daniel Goldman]

A week ago, I was processing a large input file (60,000 lines) using a 
large m4 definition file (34,000 lines). Due to an error on my part, it 
hung (forever) partway through, in an endless cycle.

My first instinct was to examine the output file for a clue to where the 
glitch was. But the output ended in the middle of a (seemingly random) 
line. m4 output to a file is buffered, so not that helpful to pinpoint 
where m4 hangs.

Using m4_traceon in the input file immediately before the incomplete 
line, I was able to figure out the glitch. But it struck me it would be 
helpful to have an m4 option to flush the output, so the user can 
quickly see exactly the last output successfully produced.

Today, the same thing happened (different input file). Again, the output 
was not very helpful localizing the glitch.

So my suggestion is to add a --flush-output command-line option to some 
future m4 version. Or at least add to the list of suggested changes.

I don't know if having the complete (flushed) output would have solved 
either of the above situations. But if I could have seen the total 
output, I could at least make my own decision.

Compared with other improvements to m4, or perhaps fundamental problems 
with m4 source versions, adding --flush-output is perhaps not real high 
priority. But it seems simple and useful in case of a glitch. And it 
seems straightforward to code (for someone who knows where in the source 
code to add fflush statements).

There are possible workarounds, but they seem awkward: *** One can at 
least turn on line flushing by not saving to a file, but that is another 
step, and then the text is only on the screen. *** To save the screen 
text, one could use "script" command, but the output is kind of messed 
up in the script file, and now we are looking at even more steps. A 
command line option seems much simpler. Maybe there is other workaround 
to flush the output I missed. If so, I'd like to hear.

A possible negative of --flush-output is adding if statement(s) will 
slow down m4, even when not flushed, but my guess is not very much. I 
did a test to investigate a little:

$ cat if-time.c
#define COUNT 50000
#define INCLUDE_IF 0

int main () {
  int i, j, k;

  for (i = 0; i < COUNT; i++) {
    for (j = 0; j < COUNT; j++) {
      k = i + (j * i);
      #if INCLUDE_IF == 1
      if (j == COUNT) {
        k = j;
        }
      #endif
      }
    }
  return 0;
  }

$ uname -s -r
Linux 3.8.0-29-generic

After compiling with INCLUDE_IF set to 0 (if NOT compiled):

$ time a.out; time a.out; time a.out

real    0m4.059s
user    0m4.056s
sys     0m0.000s

real    0m4.049s
user    0m4.044s
sys     0m0.000s

real    0m4.044s
user    0m4.040s
sys     0m0.000s

After compiling with INCLUDE_IF set to 1 (if is run):

$ time a.out; time a.out; time a.out

real    0m4.787s
user    0m4.780s
sys     0m0.000s

real    0m4.767s
user    0m4.760s
sys     0m0.000s

real    0m4.769s
user    0m4.764s
sys     0m0.000s

Adding the if statement slows things down about 17% in the above 
example. Or to put it a different way, adding 2,500,000,000 (50000 * 
50000) if statements took 0.7 seconds on my e3-1245 pc. So with a 2.5 MB 
input file, the slowdown might be 0.0007 seconds, assuming the if test 
occurs once for each character printed. The only way to know would be to 
try the change (add the if statements) in the m4 source code, and run 
some kind of benchmark.

Alternatively, there could be a m4_flush macro. But it would seem better 
to just have --flush-output command line option. It seems unlikely a 
user would just flush certain parts of the output. It would either be 
all (figure out glitch) or none (run production version).

Thanks,
Daniel