[Freeipmi-devel] Re: [llnl-devel] Re: power control and BMC configuration problem

[Albert Chu]

> I believe we are facing "OS-Absent problems with ARP" 
> See IPMI 1.5 rev1.1 - Section 12.7.1 - PDF Page# 141

That was my original belief.  But I believe "OS-Absent problems with
ARP" should exist both when the system is halted and when the system is
powered down.  This was the problem we had several weeks ago when we
could not ipmipower nodes that were powered off.

I think there is specifically some issue with bringing down an ethernet
card AND gratuitous arps AND the cisco switch configuration that is
causing this.  Ben had the idea that there may be an issue with the
cisco switch's auto-negotiation.  When the system brings the ethernet
card down, it puts the ethernet card into a funny state that negotiates
with the switch strangely.  But when the node is powered off, the
ethernet card is put into a happy state that negotiates properly with
the switch.  Cisco.com shows that some of their switches do funny things
to handle gratuitous ARP spoofing, so that could be an issue as well.
 
> I will take a look at these commands and fix them. They are supported
> by Intel's BMC. (Ian, can you take a look at this)

Just tried again, and specifically, I get an error code of CC, "invalid
data in field request" (Table 5-2 completion codes) ... 

> You have to see the ARP table of the cisco switch.

Does anyone over there know cisco switches well?  Right now we're all
mindlessly typing switch commands hoping we find something we like.

Al

--
Albert Chu
address@hidden
Lawrence Livermore National Laboratory

----- Original Message -----
From: Anand Babu <address@hidden>
Date: Thursday, January 22, 2004 3:31 pm
Subject: Re: [llnl-devel] Re: power control and BMC configuration problem

> ,----[ Albert Chu <address@hidden> ]
> | My debugging has shown that tiger4 BMCs never respond to ARP 
> requests.| When I tried to enable "BMC-generated ARP responses", 
> the BMC gave me
> | "invalid configuration option" errors.  My belief is that tiger4 
> BMCs| only send out gratuitous ARPs, and it is the job of the
> | switch/router/ethernet to see the gratuitous ARPs and cache the MAC
> | address of the card.
> `----
> 
> I believe we are facing "OS-Absent problems with ARP" 
> See IPMI 1.5 rev1.1 - Section 12.7.1 - PDF Page# 141
> 
> Possible solutions are also described in Section 12.7.2.
> 
> My recommendations:
> 1) Run a proxy ARP daemon. Robin mentioned you are already
> running such service in your network for some switches.
> 2) Gratuitous ARPs
> 
> I will take a look at these commands and fix them. They are supported
> by Intel's BMC. (Ian, can you take a look at this)
> PDF Page# 256
> - Enable BMC Generated ARP Responses
> - Enable BMC Generated Gratuitous ARP Responses
> 
> ,----[ Albert Chu <address@hidden> ]
> | > Questions: Don't these switches cache the association?
> | 
> | I'm not 100% sure.  I do know that after we halt a machine, the ARP
> | cache entries in various thunder nodes are still up to date (seen
> | through /sbin/arp -a) ...
> `----
> You have to see the ARP table of the cisco switch.
> 
> -ab
> 
> >----- Original Message -----
> >From: Anand Babu <address@hidden>
> >Date: Thursday, January 22, 2004 2:45 pm
> >Subject: Re: [llnl-devel] Re: power control and BMC configuration 
> problem
> > Hi Al,
> > 
> > I am no switch expert, just theoretical guess.
> > 
> > Condition to reproduce:
> > - when kernel halts but system is ON.
> > 
> > Crossover cable works: (reason)
> > - because it is point to point. Any packet you drop on the network
> > reaches the other end and BMC responds. (no switching or routing)
> > 
> > Cause of problem:
> > BMC does not respond to ARP queries after OS takes over. When OS 
> stops> responding to ARP requests, switch has no way to associate a 
> MAC> address to a port. 
> > 
> > Debug:
> > Cisco switch - Manually associate a MAC address to port
> > 
> > Questions:
> > Don't these switches cache the association?
> > 
> > -ab
> > 
> > >We tried the cross-over cable, and power control worked 
> perfectly. 
> > So
> > >its most likely some type of switch issue.  Have you been able to
> > >reproduce the problem??
> > 
> > 
> > _______________________________________________
> > llnl-devel mailing list
> > address@hidden
> > http://californiadigital.com/cgi-bin/mailman/listinfo/llnl-devel
> > 
> 
> 
> -- 
> Anand Babu
> Free as in Freedom <www.gnu.org>
>