Re: GNU Parallel Bug Reports niceload: macos iostat differences

[PFudd]

On Mon, 24 Sep 2012 18:50:02 +0200
Ole Tange <address@hidden> wrote:

> On Fri, Sep 21, 2012 at 8:14 PM, PFudd <address@hidden>
> wrote:
> 
> > I just tried 'niceload' for the first time today, and I like what I
> > see!
> >
> > However, it turns out that Apple has a modified version of iostat,
> > so 'niceload -I 7 ........' gives this error:
> >
> > ==============
> > iostat: illegal option -- x
> > usage: iostat [-CUdIKoT?] [-c count] [-n devs]
> >               [-w wait] [drives]
> > Use of uninitialized value $io in division (/)
> > at /usr/local/bin/niceload line 659.
> 
> What would you say is The Correct Thing to do there?

my @iostat_out = `LANG=C -x 1 2 2>&1`

if ($iostat_out[0] =~ m/iostat: illegal option -- x/) {
  die("Unsupported version of iostat found.  Please install a different
  operating system, Linux perhaps?\n");
}

> > This leaves us the unenviable task of estimating the highest
> > throughput available for arbitrary devices: yuck!  Note: the MacOS
> > Activity Monitor program has this problem too: the disk-io graph is
> > rescaled constantly, just to keep the graph's maximum value in the
> > last few minutes at about 80% of the y axis.
> 
> I do not like that way of doing it. I will prefer that niceload
> somehow replies: Unsupported feature.

That would be fine, as far as it goes.  The sysstat that's being used
by parallel is linux-specific, and even kernel-version specific:
http://sebastien.godard.pagesperso-orange.fr/faq.html#iostat
The system reports the number of ticks the disk was busy, and iostat
does the math (newticks-oldticks)/elapsedticks.

I thought the iostat from Macos might report something that could be
used for time calculations, but 'iostat -d -c 2 -w 1 -o -I' or 'iostat
-d -c 2 -w 1 -o' has a 'msps' (average milliseconds per transaction)
that's always 0.0 even under full load, so it's useless.

Perhaps niceload could have another command line option so that MacOS
users can at least specify IO bandwidth as MB/s, since they can't do
percentages. Percentages would require niceload to load that number
from a configuration file, with all of its potential for user
misconfiguration and devices randomly changing names.

> > One other note: when I tried niceload -L on my linux virtual
> > machine, I found that I didn't even have iostat installed; I don't
> > know whether niceload should print a nicer error instead of 'sh:
> > iostat: command not found/division by zero error' or if the
> > installation process should check for it; how do you feel?
> 
> The Correct Thing here would be to reply something like:
> 
>   iostat not found. iostat is required for this. Install iostat (part
> of sysstat) and retry.

Yes, during the installation process.  And for rpm and deb files to
list sysstat as a dependency, but that's for the package maintainers of
the world to take care of.  

If iostat isn't found at run time, die. Otherwise niceload -L is not
doing anything, and the system's io load will probably max out, the
system will become unusable, someone will have to debug it (ie. search
for the warning message that they previously ignored), and then curse
the authors: we sure don't want that!  :-)

> /Ole

Thanks!