sed N

[Dirk A. Froembgen]

Unix wizards,

why does sed N append a newline to any input stream that doesn't
contain one?  I'm talking about GNU sed version 3.02, even the
alpha versions.  Evidently, this is very unfortunate for text
processing with significant newlines.

I understand from the FAQ that the behavior of N in GNU sed has
been deliberately changed only recently.  I generally like the
change, except for this newline issue.

More generally, GNU sed loses track of input newlines.
For example, if I input just two lines to sed N, with only one
newline (in between), the output preserves this input exactly,
in particular, the newline.  I like that.  But if there is more
complex computation between input and output, sed appends a
newline -- which is needless at best and potentially harmful.
See my example below.

Of course, all this would go unnoticed, especially in the Unix
environment, where text streams end on a newline by convention.
But I don't like sed to enforce this convention, and in fact
it doesn't, because sed '' doesn't.

Example losing track of missing newline:

h;s/\(.*\n\)\?\(.\?\).*/\2/;x
:(
s/\n.\?//
N;G
/\n\(.\).*\n\1\|\n\n$/{s/\n[^\n]\?$//;b(}
s/\n[^\n]\?$//
P;D

Substituting $!N for N doesn't help, either, except when there
is just one input line.  Hence, it's not just a matter of N.

(The example joins all consecutive input lines that are empty
or that start with the same character.  Before joining, all
leading characters are removed, except the very first one of
every joined line.  Just a toy -- lex and awk handle that more
gracefully and more efficiently, of course.)

Sed is such a great tool.  I would very much like to have this
issue clarified.

Thank you very much,

Dirk