RE: IPtable affect or not ?? -- RE: help!! - cfservd stopsresponding after several hours

[Guolin Cheng]

Russel,

 Thanks for your suggestions. 

 Is there a fixed max. number of open connections cfservd can handle at
the same time? 

 Thanks. 
 --Guolin Cheng

-----Original Message-----
From: help-cfengine-bounces+guolin=alexa.com@gnu.org
[mailto:help-cfengine-bounces+guolin=alexa.com@gnu.org] On Behalf Of
Russell Adams
Sent: Wednesday, July 14, 2004 2:49 PM
To: help-cfengine@gnu.org
Subject: Re: IPtable affect or not ?? -- RE: help!! - cfservd
stopsresponding after several hours

This certainly could be related to your firewall setup!

Sounds like you've got alot of partially open connections lying
around, and perhaps you're not using splay time to spread out your
hosts load?

What happens when you turn off the firewall? I'd suggest that as the
first troubleshooting step. Then, after testing, add the lines back in
one at a time... Please do not do that on an internet connected host
though. ;]

Russell

On Wed, Jul 14, 2004 at 02:30:01PM -0700, Guolin Cheng wrote:
> Russel,
> 
>  "netstat -l" reports that cfengine port is listening,  while "netstat
> -ap" shows that there are hundreds of connections in "SYNC_RECV" and
> "ESTABLISHED" status.
> 
> .....
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient1.alexa.com:34584
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient2.alexa.com:34439
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient3.alexa.com:38358
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient4.alexa.com:34455
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient5.alexa.com:34558
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient6.alexa.com:60887
> SYN_RECV    -                   
> tcp        0      0 cfServer.alexa.co:cfengine
cfClient7.alexa.com:38119
> SYN_RECV    -                   
> tcp      619      0 cfServer.alexa.co:cfengine
cfClient8.alexa.co:34588
> ESTABLISHED -                   
> tcp      619      0 cfServer.alexa.co:cfengine
cfClient9.alexa.com:34675
> ESTABLISHED -                   
> tcp      619      0 cfServer.alexa.co:cfengine
cfClient10.alexa.co:34568
> ESTABLISHED -                   
> tcp      618      0 cfServer.alexa.co:cfengine
> cfClient11.alexa.com:40455  ESTABLISHED -  
> .....
> 
> That is quite strange.
> 
> I've upgraded cfengine package from 2.1.3 to current-edge 2.1.7p1
> already, and change network cable and switch port that my cfServer
> connects to, but still make no difference.
> 
> Then my only concern is:
> 
> My cfServer has iptables enabled for security: my iptables has the
> following lines in /etc/sysconfig/iptables configuration file:
> 
> ....
> :INPUT ACCEPT [0:0]
> :FORWARD DROP [0:0]
> :OUTPUT ACCEPT [0:0]
> :SCAN - [0:0]
> ....
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
> -A INPUT -m state --state INVALID,NEW -j drop-and-log-it 
> -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags FIN,RST FIN,RST -j SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags FIN,ACK FIN -j SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags PSH,ACK PSH -j SCAN 
> -A INPUT -p tcp -m tcp --tcp-flags ACK,URG URG -j SCAN 
> -A SCAN -j DROP
> ....
> 
> Any ideas? Thanks a lot.
> 
> --Guolin Cheng
> 
> 
> 
>    
> -----Original Message-----
> From: help-cfengine-bounces+guolin=alexa.com@gnu.org
> [mailto:help-cfengine-bounces+guolin=alexa.com@gnu.org] On Behalf Of
> Russell Adams
> Sent: Tuesday, July 13, 2004 12:27 PM
> To: help-cfengine@gnu.org
> Subject: Re: help!! - cfservd stops responding after several hours
> 
> What do "netstat -l" and "lsof | grep TCP" show while cfservd is hung?
> 
> Off topic tip, try using:
> 
>     ps auxw | grep [c]fservd
> 
> So you don't need a redundant grep to remove grep from the output. ;]
> 
> Russell
> 
> On Tue, Jul 13, 2004 at 12:22:05PM -0700, Guolin Cheng wrote:
> > Hi, 
> > 
> >  
> > 
> >  I got a strange problem here with cfengine 2.1.3. 
> > 
> >  
> > 
> >  The problem is: after I migration my cfengine policy server from
> > original host to a new server (change all related configurations as
> > well). The new policy server runs fine in several hours. Then
suddenly
> > it stops responding to other cfengine clients. 
> > 
> >  
> > 
> >  The symptoms are: on the client side, "cfagent -v -q" will hangs at
> the
> > stage of initially talking to cfengine policy server.
> > 
> > At the policy server end, "ps auxw | grep cfservd | grep -v grep"
> > reports no cfservd processes creation or dying. 
> > 
> >  
> > 
> >  The problem is quite strange since the policy server can respond to
> > clients very well for several hours, ( my clients run cfagent in
> > background hourly). While after several hours everything stops
> running.
> > 
> >  
> > 
> >  One response logged from the client end is attached below :
> > 
> >  
> > 
> > > 
> > 
> > >----------
> > 
> > >From:      cobalt
> > 
> > >Sent:      Tuesday, July 13, 2004 9:54:08 AM
> > 
> > >To: cobalt
> > 
> > >Subject:   Cron <root@cfClient> run-parts
> /alexa/etc/cron.d/cron.hourly
> > 
> > >Auto forwarded by a Rule
> > 
> > > 
> > 
> > /alexa/etc/cron.d/cron.hourly/cfagent.sh:
> > 
> >  
> > 
> > cfengine:: Challenge response from server
> cfServer.alexa.com/10.0.20.42
> > was incorrect!
> > 
> > cfengine:: Authentication dialogue with cfServer.alexa.com failed
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> > cfengine:cfClient: Unable to establish connection with
> > cfServer.alexa.com
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> > cfengine:cfClient: Unable to establish connection with
> > cfServer.alexa.com
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> > cfengine:cfClient: Unable to establish connection with
> > cfServer.alexa.com
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> > cfengine:cfClient: Unable to establish connection with
> > cfServer.alexa.com
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> > cfengine:cfClient: Unable to establish connection with
> > cfServer.alexa.com
> > 
> > cfengine:cfClient: Couldn't open a socket
> > 
> >  
> > 
> >  Any suggestions or opinions are greatly appreciated.  
> > 
> >  
> > 
> >  Thanks.
> > 
> >  --Guolin Cheng
> > 
> >  
> > 
> 
> > _______________________________________________
> > Help-cfengine mailing list
> > Help-cfengine@gnu.org
> > http://lists.gnu.org/mailman/listinfo/help-cfengine
> 
> 
> 
> _______________________________________________
> Help-cfengine mailing list
> Help-cfengine@gnu.org
> http://lists.gnu.org/mailman/listinfo/help-cfengine
> 


_______________________________________________
Help-cfengine mailing list
Help-cfengine@gnu.org
http://lists.gnu.org/mailman/listinfo/help-cfengine