Re: [PATCH 1/2] numa: Set default distance map if needed

[Gavin Shan]

Hi Drew,

On 10/6/21 9:35 PM, Andrew Jones wrote:
> On Wed, Oct 06, 2021 at 06:22:08PM +0800, Gavin Shan wrote:
> > The following option is used to specify the distance map. It's
> > possible the option isn't provided by user. In this case, the
> > distance map isn't populated and exposed to platform. On the
> > other hand, the empty NUMA node, where no memory resides, is
> > allowed on ARM64 virt platform. For these empty NUMA nodes,
> > their corresponding device-tree nodes aren't populated, but
> > their NUMA IDs should be included in the "/distance-map"
> > device-tree node, so that kernel can probe them properly if
> > device-tree is used.
> >
> >    -numa,dist,src=<numa_id>,dst=<numa_id>,val=<distance>
> >
> > So when user doesn't specify distance map, we need to generate
> > the default distance map, where the local and remote distances
> > are 10 and 20 separately. This adds an extra parameter to the
> > exiting complete_init_numa_distance() to generate the default
> > distance map for this case.
> >
> > Signed-off-by: Gavin Shan <gshan@redhat.com>
> > ---
> >   hw/core/numa.c | 13 +++++++++++--
> >   1 file changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/core/numa.c b/hw/core/numa.c
> > index 510d096a88..fdb3a4aeca 100644
> > --- a/hw/core/numa.c
> > +++ b/hw/core/numa.c
> > @@ -594,7 +594,7 @@ static void validate_numa_distance(MachineState *ms)
> >       }
> >   }
> >   
> > -static void complete_init_numa_distance(MachineState *ms)
> > +static void complete_init_numa_distance(MachineState *ms, bool is_default)
> >   {
> >       int src, dst;
> >       NodeInfo *numa_info = ms->numa_state->nodes;
> > @@ -609,6 +609,8 @@ static void complete_init_numa_distance(MachineState *ms)
> >               if (numa_info[src].distance[dst] == 0) {
> >                   if (src == dst) {
> >                       numa_info[src].distance[dst] = NUMA_DISTANCE_MIN;
> > +                } else if (is_default) {
> > +                    numa_info[src].distance[dst] = NUMA_DISTANCE_DEFAULT;
> >                   } else {
> >                       numa_info[src].distance[dst] = 
> > numa_info[dst].distance[src];
> >                   }
> > @@ -716,13 +718,20 @@ void numa_complete_configuration(MachineState *ms)
> >            * A->B != distance B->A, then that means the distance table is
> >            * asymmetric. In this case, the distances for both directions
> >            * of all node pairs are required.
> > +         *
> > +         * The default node pair distances, which are 10 and 20 for the
> > +         * local and remote nodes separatly, are provided if user doesn't
> > +         * specify any node pair distances.
> >            */
> >           if (ms->numa_state->have_numa_distance) {
> >               /* Validate enough NUMA distance information was provided. */
> >               validate_numa_distance(ms);
> >   
> >               /* Validation succeeded, now fill in any missing distances. */
> > -            complete_init_numa_distance(ms);
> > +            complete_init_numa_distance(ms, false);
> > +        } else {
> > +            complete_init_numa_distance(ms, true);
> > +            ms->numa_state->have_numa_distance = true;
> >           }
> >       }
> >   }
> > --
> > 2.23.0
>
> With this patch we'll always generate a distance map when there's a numa
> config now. Is there any reason a user would not want to do that? I.e.
> should we still give the user the choice of presenting a distance map?
> Also, does the addition of a distance map in DTs for compat machine types
> matter?
>
> Otherwise patch looks good to me.

Users needn't specify the distance map when the default one in kernel,
whose distances are 10 and 20 for local and remote nodes in linux for
all architectures and machines, is used. The following option is still
usable to specify the distance map.

  -numa,dist,src=<numa_id>,dst=<numa_id>,val=<distance>

When the empty NUMA nodes are concerned, the distance map is mandatory
because their NUMA IDs are identified from there. So we always generate
the distance map as this patch does :)

Thanks,
Gavin