bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] Special -i- to read one URL at a time


From: Joel Nothman
Subject: Re: [Bug-wget] Special -i- to read one URL at a time
Date: Sun, 04 Nov 2012 21:26:43 +1100
User-agent: Opera Mail/11.61 (MacIntel)


Apologies, Giuseppe. Apparently I remained in a detached head after a rebase. Pushed (and confirmed!).

- Joel


On Sun, 04 Nov 2012 21:17:40 +1100, Giuseppe Scrivano <address@hidden> wrote:

Hi Joel,

I have cloned your repository but I couldn't find that commit.  Could
you please double check if you have pushed everything?

Thanks,
Giuseppe



"Joel Nothman" <address@hidden> writes:

wgetting a list of URLs on STDIN requires reading the entire file into
memory before extracting a linked list of URLs into memory. I.e. the
STDIN  needs to have EOF before twice the input's memory is spent to
store its  URLs.

Among other things, this means wget can't deal with infinite URL lists:

$ yes http://example.com | wget -i-

I've patched wget to special-case reading from STDIN in retrieve_from_file.

Unfortunately my C's a bit rusty, and I wasn't sure how to create an
appropriate set of tests.

See https://github.com/jnothman/wget (73e9da29)

Cheers, and thanks for a great util!

- Joel



reply via email to

[Prev in Thread] Current Thread [Next in Thread]