Re: [Bug-wget] Wget can't download JPG images

[Micah Cowan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

address@hidden wrote:
> Wget Tech Support,
> 
> I am using GNU Wget 1.11.4.3287 on Windows Vista Ultimate.
> 
> I am trying to download a web comic from a website, which is a set of
> JPG images.
> 
> Here is the URL of one of the image:
> http://blacktapestries.comicgenesis.com/comics/20020711.jpg
> It's real size is 600px X 800px (85.66 KB)
> Unfortunately, every time I use Wget to download it, all I get is 1px X
> 1px (1.09 KB) image.
> It seems to be able to download JPG images from other websites, so it
> isn't a format issue.
> 
> When I attempt download all the web comic's pages from the parent
> directory, using this command:
> 
> wget --no-parent -ckr http://blacktapestries.comicgenesis.com/comics
> 
> I get this response:

<snip>

> 2009-06-01 22:00:30 ERROR 403: Forbidden.
> 
> I don't understand why this is happening because I am able to download
> the images manually using the Firefox internet browser.

The "/comics" link above is also "Forbidden" for me in a browser, so
perhaps that was a poor URL choice?

However, http://blacktapestries.comicgenesis.com/ by itself is also
forbidden for wget; adding "-U Mozilla" to wget's invocation solved that.

But to get to your actual question: the server apparently demands that
requests for images include an HTTP "Referer" header (that is, it should
be fetched because it was linked from a page, and not entered into a URL
directly). So, if you type that image's URL directly into your browser,
you will also get a 1x1 blank image, instead of the real image you can
only get by clicking a link, seeing it as part of a page, or (possibly?)
using the "view image" feature of your browser's context menu.

For wget, adding "--referer http://blacktapestries.comicgenesis.com/"; to
the invocation seems to solve the issue. I did not need to use -U
Mozilla for that, just for the actual web pages.

- --
HTH,
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer.
Maintainer of GNU Wget and GNU Teseq
http://micah.cowan.name/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkolXAMACgkQ7M8hyUobTrEagwCfe3SIP3z3H69IdRDRsOPcl2Hf
T7gAn3uBelkIitePuBFJs3S//0RHclZ8
=/kFi
-----END PGP SIGNATURE-----