Re: [PATCH v3 1/2] numa: Require distance map when empty node exists

[Igor Mammedov]

On Wed, 13 Oct 2021 12:58:04 +0800
Gavin Shan <gshan@redhat.com> wrote:

> The following option is used to specify the distance map. It's
> possible the option isn't provided by user. In this case, the
> distance map isn't populated and exposed to platform. On the
> other hand, the empty NUMA node, where no memory resides, is
> allowed on platforms like ARM64 virt. For these empty NUMA
> nodes, their corresponding device-tree nodes aren't populated,
> but their NUMA IDs should be included in the "/distance-map"
> device-tree node, so that kernel can probe them properly if
> device-tree is used.
> 
>   -numa,dist,src=<numa_id>,dst=<numa_id>,val=<distance>
> 
> This adds extra check after the machine is initialized, to
> ask for the distance map from user when empty nodes exist in
> device-tree.
> 
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> ---
>  hw/core/machine.c     |  4 ++++
>  hw/core/numa.c        | 24 ++++++++++++++++++++++++
>  include/sysemu/numa.h |  1 +
>  3 files changed, 29 insertions(+)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index b8d95eec32..c0765ad973 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -1355,6 +1355,10 @@ void machine_run_board_init(MachineState *machine)
>      accel_init_interfaces(ACCEL_GET_CLASS(machine->accelerator));
>      machine_class->init(machine);
>      phase_advance(PHASE_MACHINE_INITIALIZED);
> +
> +    if (machine->numa_state) {
> +        numa_complete_validation(machine);
> +    }
>  }
>  
>  static NotifierList machine_init_done_notifiers =
> diff --git a/hw/core/numa.c b/hw/core/numa.c
> index 510d096a88..7404b7dd38 100644
> --- a/hw/core/numa.c
> +++ b/hw/core/numa.c
> @@ -727,6 +727,30 @@ void numa_complete_configuration(MachineState *ms)
>      }
>  }
>  
> +/*
> + * When device-tree is used by the machine, the empty node IDs should
> + * be included in the distance map. So we need provide pairs of distances
> + * in this case.
> + */
> +void numa_complete_validation(MachineState *ms)
> +{
> +    NodeInfo *numa_info = ms->numa_state->nodes;
> +    int nb_numa_nodes = ms->numa_state->num_nodes;
> +    int i;
> +
> +    if (!ms->fdt || ms->numa_state->have_numa_distance) {

also skip check/limitation when VM is launched with ACPI enabled?

> +        return;
> +    }
> +
> +    for (i = 0; i < nb_numa_nodes; i++) {
> +        if (numa_info[i].present && !numa_info[i].node_mem) {
> +            error_report("Empty node %d found, please provide "
> +                         "distance map.", i);
> +            exit(EXIT_FAILURE);
> +        }
> +    }
> +}
> +
>  void parse_numa_opts(MachineState *ms)
>  {
>      qemu_opts_foreach(qemu_find_opts("numa"), parse_numa, ms, &error_fatal);
> diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h
> index 4173ef2afa..80f25ab830 100644
> --- a/include/sysemu/numa.h
> +++ b/include/sysemu/numa.h
> @@ -104,6 +104,7 @@ void parse_numa_hmat_lb(NumaState *numa_state, 
> NumaHmatLBOptions *node,
>  void parse_numa_hmat_cache(MachineState *ms, NumaHmatCacheOptions *node,
>                             Error **errp);
>  void numa_complete_configuration(MachineState *ms);
> +void numa_complete_validation(MachineState *ms);
>  void query_numa_node_mem(NumaNodeMem node_mem[], MachineState *ms);
>  extern QemuOptsList qemu_numa_opts;
>  void numa_cpu_pre_plug(const struct CPUArchId *slot, DeviceState *dev,