Re: [Freeipmi-users] ipmiconsole: "could not set errnum: current = 31, desired = 25"

[Albert Chu]

Hi David,

On Mon, 2014-08-11 at 10:07 +1000, David O'Shea wrote:
> Hi all,
> 
> With FreeIPMI 1.4.5, using libipmiconsole from Python to talk to a Dell 
> iDRAC, I've had intermittent errors.  To date, it normally seems like when 
> libipmiconsole reports a timeout, when I look at the host's virtual console 
> in the iDRAC web interface, I see evidence of other network issues (whether 
> the fault of the iDRAC or my network, I don't know).  I  think I've seen 
> errors like that roughly 3% of the times I run my script.
> 
> However, I just hit an issue that seems to (a) have only affected IPMI
> SOL and (b) involves an internal libipmiconsole error.  After IPMI SOL
> had been connected for about 6 minutes and about 800 lines of text had
> been sent, I got these messages:
> 
> (ipmiconsole_processing.c, _sol_bmc_to_remote_console_packet, 2801):
> hostname=silicon-bmc.adl.quantum.com; protocol_state=9h: scbuf_write:
> dropped data: dropped=14
> (ipmiconsole_processing.c, _process_ctx, 4005):
> hostname=silicon-bmc.adl.quantum.com; protocol_state=Ah: closing
> session due to session timeout
> (ipmiconsole_ctx.c, ipmiconsole_ctx_set_errnum, 1396): could not set
> errnum: current = 31, desired = 25

Interesting, it appears that the buffer (scbuf_write) overflowed its
max, which is defaulted to 16K.  That's what likely caused the internal
error.  I guess there was a ton of data coming out.  It's possible if
the buffer were increased you may not see the problem anymore.  The max
was picked along time ago and perhaps BMCs are simply able to pump out
more data than they used to.

> I gather the last message indicates an attempt to set the errnum to 
> IPMICONSOLE_ERR_SESSION_TIMEOUT when it has already been set to 
> IPMICONSOLE_ERR_INTERNAL_ERROR earlier, and this failed because the previous 
> errnum hadn't been read yet?

Yup.

> The strange thing is I would have expected the earlier _INTERNAL_ERROR to 
> cause an EOF, but even the timeout didn't cause one immediately - I kept 
> receiving data from the file descriptor for another 2 seconds before I 
> received an EOF, although admittedly from the timestamps in my log, which 
> include milliseconds, it looks like the data was coming fairly continuously 
> and therefore may have been buffered.

That's a very likely scenario.

> Am I using libipmiconsole incorrectly if I'm just reading from the file 
> descriptor waiting for EOF to indicate an error?

Nope, that's correct usage.  Your situation is admittedly unique.  Is
this something that can't be programmed around?  Or is this more just a
"interesting thing that happened."

> I configure libipmiconsole with IPMICONSOLE_DEBUG_STDERR; should the previous 
> _INTERNAL_ERROR have been accompanied by a message to stderr?  I couldn't 
> find one.

The output you show above it the kind of debugging output you should
see.  The INTERNAL_ERROR was likely caused by the
_sol_bmc_to_remote_console_packet message.  The debug messages don't
necessarily tell you they are related to any particular error code.
Probably could have added more, but I guess the debugging code was more
for people who knew the code well :P

Hope that answers your questions,

Al

> If I enabled verbose tracing, would that possibly help to determine why I got 
> the internal error and/or session timeout?  It would be good if I could 
> understand a little more about the issue so I could report it to Dell.
> 
> I gather there is no way to provide a logging callback to libipmiconsole and 
> I just have to choose from one of the following options?
> 
>  * STDOUT       - Output debugging to stdout
>  * STDERR       - Output debugging to stderr
>  * SYSLOG       - Output debugging to the Syslog
>  * FILE         - Output debugging to files in current working directory
> 
> Thanks in advance,
> David
> 
> 
>                                         
> _______________________________________________
> Freeipmi-users mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/freeipmi-users
-- 
Albert Chu
address@hidden
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory