[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] How do I use relative files on the cmd line with a base?

From: Linda Walsh
Subject: Re: [Bug-wget] How do I use relative files on the cmd line with a base?
Date: Fri, 15 Jun 2012 10:50:52 -0700
User-agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv: Gecko/20100228 Lightning/0.9 Thunderbird/ Mnenhy/

Cngel GonzC!lez wrote:
On 14/06/12 13:19, Linda Walsh wrote:
I have a list of files in a directory.

I only want each wget to retrieve 10 of the files, so I want to use
parallel to put 10 of the files on wget's command line.

I tried specifying the base with -B, but it seems to ignore it.

how do I specify a base with files on the command line?

I've wanted to do this before ..... where several files I want to fetch
are in 1 dir, and I only want to type the dirname 1 time, so how does
one do this in wget?

I'm not sure if this is a question on splitting a list of urls in groups
of 10, or about not typing the domain for each url (paste command
doesn't work?).

Assuming you are using a unix-like shell, you can use:
 wget http://example.org/path/{file1,file2,subfolder/file3,file4}
and that will expand being equivalent to:
 wget http://example.org/path/file1 http://example.org/path/file2
http://example.org/path/subfolder/file3 http://example.org/path/file4

well -- the http address is 80 characters long --
some of the base's pathname chars had  shell unfriendly
chars in them -- to the point that I *had* to put the base URL into
a file and not on the command line (kept running into quoting problems)...

The filenames were relatively simple (foo.rpm... x.rpm...etc...)..
Try your example with base="http://download.openproject.org/repositories/devel:/languages:/perl/openproject_Factory/src/?release=alpha1.0&lang=en&encoding=binary&source="; and files = package?arch=x86_64&fmt=rpm,name=package1 package?arch=no_arch&fmt=rpm,name=package2, ...

Expand that in a shell... and it doesn't work so well...

Not to mention, doing 10 lines at a time make the lines expand to
over 1360 chars (because the base is 136 chars by itself
exclusive of wget options and the file names...

So I wanted to put the base in a file and just use -B "$(<base)", and put the filenames at the end. -- and not use the shell but use 'parallel', which spawns processes directly, so I don't have to worry about expansion -- (I could have it do expansion by running it with bash -c "command", but if expansion is causing a problem
its a bonus not to incur running a command shell with each iteration)....

Many times, I've had a long URL name that I want to set, then just enter filenames relative to that base.
and with the base being complicated -- I just want to put it in a file...)

I don't see why putting filenames on the command line ignores the -B switch??

I.e. -- what is the purpose served by ignoring a "B" switch the user specifies on the command
line when they fetch files?  Is there a benefit to it being ignored?

reply via email to

[Prev in Thread] Current Thread [Next in Thread]