Re: [Bug-wget] Anyone want to add libcares support to wget?

[Ben Greear]

On 03/18/2016 02:33 AM, Tim Ruehsen wrote:
> Tim
>
> On Thursday 17 March 2016 09:36:47 Ben Greear wrote:
> > On 03/17/2016 08:58 AM, Giuseppe Scrivano wrote:
> > > Hi Ben,
> > >
> > > Ben Greear <address@hidden> writes:
> > > > > Did you consider using a container (e.g. docker) for such a task ? Easy
> > > > > to set up and you'll have your feature not only for wget. IMO, that is
> > > > > much more flexible. (It was Giuseppe's idea during a private talk).
> > > >
> > > > Containers will not work for me.  I need to scale to thousands of
> > > > instances on modest hardware.  I'm certain the libcares and binding
> > > > approach will work because we do similar things with curl and other
> > > > programs already.
> > >
> > > how modest is this hardware?
> > >
> > > I tried to build a minimal Docker container for wget on top of Alpine:
> > >
> > > Dockerfile:
> > >
> > > FROM alpine
> > > RUN apk add --update wget
> > > WORKDIR /out
> > > ENTRYPOINT ["/usr/bin/wget"]
> > >
> > > and simply running (192.168.1.13 is an internal DNS server):
> > >
> > > docker run --dns=192.168.1.13 --rm -v $(pwd):/out:Z wget wdserver
> > >
> > > takes around 5M, plus the docker daemon.
> > >
> > > If this is not enough, you can even just run wget in a chroot, and
> > > provide a different /etc/resolv.conf.
> >
> > I really am not at all interested in LD_PRELOAD, docker, chroot, or any
> > other such a thing.  I have 15+ years invested in my current architecture,
> > and it works very well for me.  It absolutely cannot be easier to manage
> > docker instances on random platforms vs using a few cmd-line switches to
> > wget.
> >
> > I know that this feature will not be useful for most people, but then again,
> > none of us can know what some user might be able to do with such a feature.
> >   I know that some users of curl + libcares wanted the ability to specify
> > certain DNS servers to get around oppressive firewalls (on systems they
> > could not have root privs), for instance.
> >
> > I don't mind making the changes to wget.  I thought if someone was
> > interested in a project, then I'd be happy to work with them (and pay them,
> > etc).
>
> Why not put an offer here (with an exact description of what to deliver, OS,
> etc) ?

A lot of lists do not like such postings, so I was trying to be a bit discreet.

Anyway, I'll pay $500 USD for a working implementation, and $1000 if someone
can get it coded up well enough that it is accepted upstream.  If someone thinks
this is not an appropriate amount, let me know your suggestions.

Features:

 * Ability to link with libcares for all DNS lookup requests.
 * Ability to specify DNS server list for libcares.  At least IPv4.  IPv6 is
   nice but not required.
 * Ability to bind libcares to a local IP address.  Ability to also bind to 
local
   network interface is nice but not required.  libcares library already 
supports this,
   see 'curl' for examples of how this was done.
 * All of this configurable with cmd-line arguments.
 * Ability to specify which libcares to include and link against (with 
./configure option or similar)

 * OS:  Linux, modern Fedora.

> > I'll post my patches for consideration, and you will be welcome to pull it
> > into your tree.  If they are not acceptable, then I can maintain the fork
> > for as long as my customers care to use it..
>
> Your patches are welcome, no doubt.
>
> Just please understand our position that we try to find solutions by other
> means before introducing new options.

I understand, and I don't mind the suggestions...its just that in this case,
I am pretty certain what I need and want, and some of the more generic 
work-arounds
suggested just do not meet my needs.

Thanks,
Ben

> Regards, Tim


--
Ben Greear <address@hidden>
Candela Technologies Inc  http://www.candelatech.com