Re: lynx-dev Revised patch for HTFTP.c

[Doug Kaufman]

On Sat, 19 Aug 2000, Klaus Weide wrote:

> On Fri, 18 Aug 2000, Doug Kaufman wrote:
> 
> > 1. ASCII mode for the ftp protocol is designed to transfer text files
> > between machines with different native formats, converting the formats
> > as transfer is done.
> > 
> > 2. The FTP server can not determine the format of the files.
> 
> Of course an FTP server *could* analyze the contents of a file that
> is local (to it) and perhaps vary its actions accordingly.  I don't
> know of any that do.

Exactly. If no known FTP server does this, we should not expect
servers to do it, even if theoretically possible.

> > 4. Lynx can render text files that have EOLs for Macintosh (CR),
> > DOS (CR LF), or unix (LF).
> 
> I regard this more as an accident than anything else.  An accidental

Whether intentional or accidental, the bottom line is that lynx
doesn't need the text conversion that is done by the server during
ascii transfers, except when dealing with a server that does not use
Macintosh, DOS, or unix EOL conventions.

> > 5. ASCII transfer mode would be appropriate when transfering text
> > files from servers whose native mode is not Macintosh, DOS, or unix
> > (e.g., VM/CMS).
> 
> ASCII transfer mode is appropriate when transferring text files from
> ANY servers, independent of their native mode (or even whether they
> have one).  That's what ASCII mode is meant for.

It was only meant for this if you want the "text" file converted from
one native mode to that of another machine. Why would lynx want to
do that when it can handle the unmodified file, as it resides on the
server?
  
> Now it happens that for some combinations of server and client systems,
> binary (image) mode can *also* be use for transferring text files, such
> ... 
> But these are _logically_ the exceptions - no matter that numerically,
> they may cover the vast majority of cases encountered in practice.  
> Relying on this for text transfer means relying on an accidental
> similarity between sender and recipient native text representations.
> 
> You realize that this isn't always the case.  But, if I understand you
> right, you want to specially recognize the cases where the similarity does
> not occur ("e.g., VM/CMS") and treat everything else, by default, under
> a "similar enough" assumption.  I could find a more conservative approach
> acceptable - make the "similar enough" assumption only if we have concrete
> indication that the server system is indeed similar to the client's; maybe
> by virtue of the server explicitly telling us that it's Macintosh, DOS, or
> unix.  But I don't think it would be very useful.

Lynx already does binary transfer of all files for the cases of
download, -source, and -dump, except when server=CMS_SERVER.
  
> The FTP protocol ought to be general enough such that it will continue
> to work in the future, with currently not yet existing systems, as long
> ... 
> If I understand yout patch right, if it got applied now, lynx would
> not work correctly (for simple text rendering) with those systems,
> until it gets specifically patched to work with them.  (I am assuming

I think that is true. Yet we already have lots of server-specific
code in HTFTP.c. I doubt that a new type of server would work in the
general case, but it certainly could. It would be easy to enumerate
the servers for which we could default to binary transfer, rather than
my proposal, which was for all that were not CMS_SERVER. Aside from
CMS_SERVER and perhaps GENERIC_SERVER, are there any server types in
the eServerType enum for which lynx doesn't properly render "text"
files transferred in binary mode?

> > 6. We can not count on the fact that files that appear to be text
> > will always be in the native format of the server, although it is
> > apparently common practice for this to be true.
> 
> >From which perspective is this, i.e. who is "we"?  If "we" is just the FTP
> client side, then it shouldn't matter to us at all how the server *stores*
> file.  All that matters is how it sends them to us, since that is all we
> are going to see.

But the server sends the file whichever way we request, either binary
or ascii. As the client, we are making assumptions about which
files should be transferred in ascii mode. Some real life examples
have proven that these assumptions are occasionally wrong. I simply
proposed changing the assumptions.
  
> > It appears to me that the patch fixes the problem with extra lines
> > appearing in rendering of DOS style files on unix servers, without
> > breaking anything. If no one sees anything that gets broken by the
> > patch, I would still recommend its incorporation into the lynx code.
> 
> I see it breaking the generality of the protocol assumptions.  It
> anticipates only that which is currently observed in practice, as opposed
> to: the full range of possible situations for which the protocol
> allows.

I think we disagree here, although I am willing to be convinced.
How does it change the generality of the protocol to change the
default behavior, when both transfer modes are still available for any
particular file (by adding ";type=[AI]" to the URL)? FTP assumes that
you know what type of file you are retrieving and use the appropriate
method. I would hold that a so-called text file, when not in the
native format of the server on which it resides, is actually a binary
file for that server, as much as is a binary executable file. When a
binary executable is retrieved in ascii mode, it comes as a broken
file. I believe that we are doing the same with non-native format
"text" files.
  
> Compare the following two statements:
> 
>   All systems that matter these days are ASCII based, of either the
> ... 
>   All browsers that matter these days are configured to show inline
>   images / frames / interpret Javascript / ...  Actually there are some

I agree that a generic solution is preferable, but, considering the
number of users complaining over the years of corrupt files on ftp
servers because they mistakenly retrieved binary files in the default
ascii mode, I doubt that such a generic solution can be readily
implemented.
                               Doug
__ 
Doug Kaufman
Internet: address@hidden




; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden