[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support
From: |
Igor Mammedov |
Subject: |
Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support |
Date: |
Wed, 9 Mar 2016 11:48:41 +0100 |
On Wed, 9 Mar 2016 13:58:53 +1100
David Gibson <address@hidden> wrote:
> On Tue, Mar 08, 2016 at 10:37:08AM +0100, Igor Mammedov wrote:
> > On Tue, 8 Mar 2016 15:27:39 +1100
> > David Gibson <address@hidden> wrote:
> >
> > > On Mon, Mar 07, 2016 at 11:59:42AM +0530, Bharata B Rao wrote:
> > > > On Mon, Mar 07, 2016 at 02:49:06PM +1100, David Gibson wrote:
> > > > > On Fri, Mar 04, 2016 at 12:24:19PM +0530, Bharata B Rao wrote:
> > > > > > Set up device tree entries for the hotplugged CPU core and use the
> > > > > > exising EPOW event infrastructure to send CPU hotplug notification
> > > > > > to
> > > > > > the guest.
> > > > > >
> > > > > > Signed-off-by: Bharata B Rao <address@hidden>
> > > > > > ---
> > > > > > hw/ppc/spapr.c | 73
> > > > > > ++++++++++++++++++++++++++++++++++++++++-
> > > > > > hw/ppc/spapr_cpu_core.c | 60
> > > > > > +++++++++++++++++++++++++++++++++
> > > > > > hw/ppc/spapr_events.c | 3 ++
> > > > > > hw/ppc/spapr_rtas.c | 24 ++++++++++++++
> > > > > > include/hw/ppc/spapr.h | 4 +++
> > > > > > include/hw/ppc/spapr_cpu_core.h | 2 ++
> > > > > > 6 files changed, 165 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > > index 5acb612..6c4ac50 100644
> > > > > > --- a/hw/ppc/spapr.c
> > > > > > +++ b/hw/ppc/spapr.c
> > > > > > @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState
> > > > > > *cs, void *fdt, int offset,
> > > > > > size_t page_sizes_prop_size;
> > > > > > uint32_t vcpus_per_socket = smp_threads * smp_cores;
> > > > > > uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> > > > > > + sPAPRMachineClass *smc =
> > > > > > SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > > > > > + sPAPRDRConnector *drc;
> > > > > > + sPAPRDRConnectorClass *drck;
> > > > > > + int drc_index;
> > > > > > +
> > > > > > + if (smc->dr_cpu_enabled) {
> > > > > > + drc =
> > > > > > spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> > > > > > + g_assert(drc);
> > > > > > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > > > > > + drc_index = drck->get_index(drc);
> > > > > > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index",
> > > > > > drc_index)));
> > > > > > + }
> > > > > >
> > > > > > /* Note: we keep CI large pages off for now because a 64K
> > > > > > capable guest
> > > > > > * provisioned with large pages might otherwise try to map a
> > > > > > qemu
> > > > > > @@ -987,6 +999,16 @@ static void
> > > > > > spapr_finalize_fdt(sPAPRMachineState *spapr,
> > > > > > _FDT(spapr_drc_populate_dt(fdt, 0, NULL,
> > > > > > SPAPR_DR_CONNECTOR_TYPE_LMB));
> > > > > > }
> > > > > >
> > > > > > + if (smc->dr_cpu_enabled) {
> > > > > > + int offset = fdt_path_offset(fdt, "/cpus");
> > > > > > + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > > > > > + SPAPR_DR_CONNECTOR_TYPE_CPU);
> > > > > > + if (ret < 0) {
> > > > > > + error_report("Couldn't set up CPU DR device tree
> > > > > > properties");
> > > > > > + exit(1);
> > > > > > + }
> > > > > > + }
> > > > > > +
> > > > > > _FDT((fdt_pack(fdt)));
> > > > > >
> > > > > > if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> > > > > > @@ -1181,7 +1203,7 @@ static void ppc_spapr_reset(void)
> > > > > >
> > > > > > }
> > > > > >
> > > > > > -static void spapr_cpu_reset(void *opaque)
> > > > > > +void spapr_cpu_reset(void *opaque)
> > > > > > {
> > > > > > sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > > > PowerPCCPU *cpu = opaque;
> > > > > > @@ -1622,6 +1644,8 @@ static void spapr_boot_set(void *opaque,
> > > > > > const char *boot_device,
> > > > > > void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> > > > > > Error **errp)
> > > > > > {
> > > > > > CPUPPCState *env = &cpu->env;
> > > > > > + CPUState *cs = CPU(cpu);
> > > > > > + int i;
> > > > > >
> > > > > > /* Set time-base frequency to 512 MHz */
> > > > > > cpu_ppc_tb_init(env, TIMEBASE_FREQ);
> > > > > > @@ -1646,6 +1670,14 @@ void spapr_cpu_init(sPAPRMachineState
> > > > > > *spapr, PowerPCCPU *cpu, Error **errp)
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > + /* Set NUMA node for the added CPUs */
> > > > > > + for (i = 0; i < nb_numa_nodes; i++) {
> > > > > > + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> > > > > > + cs->numa_node = i;
> > > > > > + break;
> > > > > > + }
> > > > > > + }
> > > > > > +
> > > > >
> > > > > This hunk seems like it belongs in a different patch.
> > > >
> > > > It appears that this would be needed by other archs also to set the
> > > > NUMA node for the hot-plugged CPU. How about make an API out of this
> > > > and use this something like below ? Igor ?
> > >
> > > Is there a way we could put this in the the CPU thread initialization
> > > itself? Rather than requiring every platform to call a helper.
> > I'd suggest hotplugable CPU entity to have 'node' property, like we have
> > in pc-dimm.
>
> Ok. Do you think that makes sense for the base core type (which will
> sometimes be a hotpluggable CPU entity and sometimes might not be)?
I wouldn't push it into generic base type, as in general target
implementing it in subtype might not have a numa concept at all.
I'd be better to leave decision up to target and corresponding subtype.
>
> > However machine owns numa mapping, so setting it from thread
> > initialization seems to be wrong.
>
> Ah.. good point.
>
> > Could that be done from machine's plug() handler (spapr_core_plug)?
>
> Yes, I think so. The core might need a "set node" method, but we
> should be able to control this from the plug() handler.
>
> > Also I notice that there is in no way to check/set 'address' properties
> > at machine level before calling cpu->realize(), which makes us
> > to attempt checking them inside cpu->realize() or at post realize time
> > in HotplugHandler(machine)->plug() hook with following full rollback
> > in case of failure.
> > Maybe we need add an HotplugHandler->pre_realize() hook to allow machine
> > verify/set properties of CPU before it's realized.
>
> I don't really see what the problem is with checking inside realize().
pls see http://patchwork.ozlabs.org/patch/594196/ for reasoning behind this
- [Qemu-devel] [RFC PATCH v1 04/10] cpu: Add a sync version of cpu_remove(), (continued)
- [Qemu-devel] [RFC PATCH v1 04/10] cpu: Add a sync version of cpu_remove(), Bharata B Rao, 2016/03/04
- [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Bharata B Rao, 2016/03/04
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, David Gibson, 2016/03/06
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Bharata B Rao, 2016/03/07
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Igor Mammedov, 2016/03/07
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, David Gibson, 2016/03/07
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Igor Mammedov, 2016/03/08
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, David Gibson, 2016/03/08
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Bharata B Rao, 2016/03/09
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support, Igor Mammedov, 2016/03/09
- Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support,
Igor Mammedov <=
[Qemu-devel] [RFC PATCH v1 07/10] spapr: Represent boot CPUs as spapr-cpu-core devices, Bharata B Rao, 2016/03/04
[Qemu-devel] [RFC PATCH v1 06/10] spapr: CPU core device, Bharata B Rao, 2016/03/04
[Qemu-devel] [RFC PATCH v1 09/10] xics, xics_kvm: Handle CPU unplug correctly, Bharata B Rao, 2016/03/04
[Qemu-devel] [RFC PATCH v1 10/10] spapr: CPU hot unplug support, Bharata B Rao, 2016/03/04
Re: [Qemu-devel] [RFC PATCH v1 00/10] Core based CPU hotplug for PowerPC sPAPR, Igor Mammedov, 2016/03/04