Re: [Gluster-devel] CLIENT_PORT_CIELING (sic)

[Geoff Kassel]

Hi Raghavendra,
   I'm using just 4 protocol/client translators across two servers. I found 
after posting my first message why it was starting so many client 
connections - there was an error in the name of remote brick a client was 
trying to access, causing it to keep trying to connect over and over again. 
(And hence take privileged port after privileged port.)

   But my main point about port usage still remains.

   I'm aware that GlusterFS doesn't bind to ports already in use - that's not 
the problem. The problem is that it uses ports that should be reserved for 
other services, and there's not a lot of room for newer applications to start 
taking privileged port numbers at random.

   The services I'm using depend on GlusterFS - meaning these services can 
only be successfully started after GlusterFS starts. Hence those ports are 
not in use when GlusterFS starts. GlusterFS takes them for its own - without 
regard for previous IANA assignment - which is not desirable behaviour, as 
this stops these other services from being able to start.

   Also, it seems that any form of authentication over a TCP socket triggers 
this issue - I've tried authentication by password (because the documentation 
strongly implied that this would not force the use of privileged ports) and 
it still demonstrated this port usage pattern. (I later had to modify 
PRIVILAGED_PORT_CIELING - also misspelled - to get a client connecting to a 
server from a non-privileged port, by the way.)

   I can post the erroneous configuration that triggered the many connection 
attempts, if you like - or the working version. (Working after a fashion, as 
it had problems which caused me to revert to a more stable configuration, but 
I'll get into that later.) Which would you prefer?

Kind regards,

Geoff Kassel.

On Wed, 4 Feb 2009, Raghavendra G wrote:
> Hi,
>
> How many protocol/client translators are you using? Can you mail the volume
> specification file? glusterfs tries to bind to privileged ports which are
> free. It cannot bind to ports which are already being used. No need to
> change the value of CLIENT_PORT_CEILING, since if the client cannot bind to
> any privileged ports it will try to bind to non privileged ones. But If the
> authentication being used on server side is ip based, it requires client to
> be bound to a privileged port.
>
> regards,
> On Tue, Feb 3, 2009 at 6:26 PM, Geoff Kassel
>
> <address@hidden>wrote:
> > Hi,
> >   Noticed an odd thing with GlusterFS 2.x. It (as compared to 1.3.8) now
> > takes over a lot of privileged ports - and a lot of times, the ports
> > stolen aren't really 'free' for GlusterFS to be using.
> >
> >   For example, I run mail servers, and provide the usual access methods -
> > POP3, IMAP, and their SSL variants on ports 993 and 995. I've found that
> > GlusterFS will often take over ports 993 and 995 - and about twenty other
> > privileged ports at the same time. Sometimes really low ports, like port
> > 1 (!)
> >
> >   This stops other services from binding to these ports. Hence, when I
> > attempt a full restart of my GlusterFS dependent services (i.e. when
> > changing
> > GlusterFS configuration, or recovering from a crash) I find most of my
> > services won't restart properly, which is a big issue for me.
> >
> >   Browsing through the code, I found the CLIENT_PORT_CIELING (sic - that
> > should probably be CLIENT_PORT_CEILING) setting in the transport
> > translators,
> > and I've altered this appropriately up to 65536, so that there's no
> > competition between GlusterFS and my existing services.
> >
> >   However, I feel that this is a somewhat inconvenient step to have to
> > make in order to get GlusterFS to play nice with other services.
> >
> >   Would it be possible to have a translator configuration option that
> > allows
> > this to be set from a server or client volume file, without having to
> > manually patch and recompile GlusterFS each time I want to change this
> > setting?
> >
> >   Thank you in advance for your time and effort in considering my
> > request.
> >
> > Kind regards,
> >
> > Geoff Kassel.
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel