Re: [PATCH v2 2/6] spapr_numa: forbid asymmetrical NUMA setups

[Daniel Henrique Barboza]

On 9/26/20 4:49 AM, David Gibson wrote:
> On Fri, Sep 25, 2020 at 09:41:02AM -0300, Daniel Henrique Barboza wrote:
> > On 9/25/20 12:48 AM, David Gibson wrote:
> > > On Thu, Sep 24, 2020 at 04:50:54PM -0300, Daniel Henrique Barboza wrote:
> > > > The pSeries machine does not support asymmetrical NUMA
> > > > configurations. This doesn't make much of a different
> > > > since we're not using user input for pSeries NUMA setup,
> > > > but this will change in the next patches.
> > > >
> > > > To avoid breaking existing setups, gate this change by
> > > > checking for legacy NUMA support.
> > > >
> > > > Reviewed-by: Greg Kurz <groug@kaod.org>
> > > > Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> > >
> > > Having read the rest of the series, I realized there's another type of
> > > configuration that PAPR can't represent, so possibly we should add
> > > logic to catch that as well.  That's what I'm going to call
> > > "non-transitive" configurations, e.g.
> > >
> > > Node    0       1       2
> > > 0       10      20      40
> > > 1       20      10      20
> > > 2       40      20      10      
> > >
> > > Basically the closeness of 0 to 1 and 1 to 2 forces them all to be in
> > > the same domain at every PAPR level, even though 0-2 is supposed to be
> > > more expensive.
> >
> > Yes, this is correct. I'm not sure how to proceed in this case
> > though. Should we error out?
>
> Given that we're already erroring on asymmetric configurations, I
> think it makes sense to error for these as well.

Thing is that asymmetrical configurations is an easy concept to enforce
to the user - distance from A to B can't be different from B to A.

In the example you gave above, with 3 NUMA nodes, is easy to spot where
the non-transitivity rule would hit. I'm afraid that if we add 2-3 more
NUMA nodes in the mix this will stop being straightforward, with more and
more combinations hitting the 'non-transitivity' rule, and erroring out
will end up being frustrating to the user.

I'd say that we should report this in the documentation as one more
limitation of the implementation (and PAPR). I wouldn't oppose with
throwing a warning message either, letting the user know that the
approximation will be less precise than it already would be in this case.


Thanks,


DHB



>