Re: lynx-dev Other uses for <!-- X-URL --> and <BASE HREF>

[Klaus Weide]

On Mon, 19 Jun 2000, pAb-032871 wrote:

> More or less hypothetical but this might come in handy later,
> if/when I learn more about programming and Lynx's code.
> 
> The automatic insertions of;
> <!-- X-URL: http ://www.server.dom/~username/ -->
> <BASE HREF="http ://www.server.dom/~username/">
> are very useful, keeping relative links in dowloaded/saved HTML
> current [spaces added to avoid bad links in mail archive].

Not everyone agrees that this is a good thing; in the latest
Debian lynx package, the maintainer has disabled this behavior
by default (see PREPREND_* in lynx.cfg), after a complaint that
"Lynx download mangles html files".

> How hard would it be to add this to text/plain files as well?

Probably not hard.

But quite different in effect - in text/html, this information is
hidden away, in text/plain there's no way to hide it.

I also think it is dangerous, for downloaded files that are actually
some binary type but mislabeled as text/plain.  The binary file would
be corrupted.  I have seen erroneous labeling as text/plain much more
often than erroneous labeling as text/html.

> Might seem like a strange thing to do, useful only to absent-minded
> twits like me who save interesting things from the web and then
> forget where they came from. . .
> 
> Adding something like;
>     Linkname: Page Title
>          URL: http ://www.server.dom/~username/
> would be prettier, but the X-URL/BASE HREF thing seems easier
> because it's already set up to do this with text/html.  I'm guessing
> Linkname/URL would require borrowing some elements from the "="
> info-page and generally make things too complicated.

In non-interactive mode, you can use -crawl (in addition to -dump,
*no* -traversal) to get rendered text prefixed with two lines like
   THE_URL:http://localhost/
   THE_TITLE:Welcome to Your New Home Page!
This also seems to work for text/plain, except that there is no TITLE.

You could make lynx call itself with -dump -crawl as an EXTERNAL command,
if you often want to save plain text files in this way.

To save a different kind of meta information together with a file,
you can use -mime_header.  Again, this could be used as an EXTERNAL.
But the URL is not part of that info (headers of HTTP response).

The best way to add meta information like this to downloaded files
would be to create a specialized DOWNLOADER script, which could add
anything it likes to the file.  The problem is that a downloader
command currently has no access to the information (URL, title, ...).
Lynx could set some environment variables to make the information
available when a DOWNLOADER is executed, as it already does for
PRINTER commands.  (IOW, you can already do something like this
if you use 'P' not 'd').

> Yes, the simplest thing would be to save info-pages along with
> text files, or to save source instead.  I just got curious, and
> this seemed like a good time to ask since list traffic is pretty
> slow right now.

I wasn't aware that lynx ever prepends the "<!-- X-URL ...>" and
<BASE ...>" stuff for text/plain documents, but apparently it does
if one uses 'P'rint from source view!  Just switch using '\' (which
shouldn't change the appearance at all, but the statusline will
indicate source view), then 'P'.  Now this is probably unintentional
(overlooked because source view of plain text isn't really meaningful),
but seems to be exactly what you want.

> By the way, what's the "X-URL:" comment for?  Just something Lynx
> throws in to show this wasn't originally in the page source? 
> A provision for times when the BASE is different from the actual
> location of the document?

"X-URL:" was there first, long before BASE was added IIRC.
I think it made sense to keep it, given that the BASE URL can
be different as you note.

   Klaus


; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden