Re: [EXT] Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Shesha Bhushan Sreenivasamurthy]

From: Ira Weiny <ira.weiny@intel.com>
Sent: Friday, June 9, 2023 2:06 PM
To: nifan@outlook.com <nifan@outlook.com>; Ira Weiny <ira.weiny@intel.com>; 
Shesha Bhushan Sreenivasamurthy <sheshas@marvell.com>
Cc: Shesha Bhushan Sreenivasamurthy <sheshas@marvell.com>; Fan Ni 
<fan.ni@samsung.com>; Jonathan Cameron <Jonathan.Cameron@huawei.com>; 
qemu-devel@nongnu.org <qemu-devel@nongnu.org>; linux-cxl@vger.kernel.org 
<linux-cxl@vger.kernel.org>; gregory.price@memverge.com 
<gregory.price@memverge.com>; hchkuo@avery-design.com.tw 
<hchkuo@avery-design.com.tw>; cbrowy@avery-design.com 
<cbrowy@avery-design.com>; dan.j.williams@intel.com <dan.j.williams@intel.com>; 
Adam Manzanares <a.manzanares@samsung.com>; dave@stgolabs.net 
<dave@stgolabs.net>; nmtadam.samsung@gmail.com <nmtadam.samsung@gmail.com>
Subject: [EXT] Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu 
 
External Email

----------------------------------------------------------------------
nifan@outlook.com wrote:
> The 06/08/2023 08:43, Ira Weiny wrote:
> > Shesha Bhushan Sreenivasamurthy wrote:

[snip]

> 
> Hi Ira & Shesha,
> FYI. I reabased my patch series on top of the above branch and created a new
> branch here:
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_moking_qemu-2Ddcd-2Dpreview-2Dlatest_tree_dcd-2Dpreview&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=Zta64bwn4nurTRpD4LY2OGr8KklkMRPn7Z_Qy0o4unU&m=SvyB_49EIFUT8-ZEVgIEYYjU6-zGTX4wb30kuNLUhkTSHYZK5-C0Gxr7uvefhtj4&s=MFD7qlSaTuy-w6aDmavIMbSP_aeaqZmSML7IVOX5jLs&e=
>  

Thanks!

> 
> It passes the same tests as shown here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_linux-2Dcxl_6481f70fca5c2-5Fc82be29440-40iweiny-2Dmobl.notmuch_T_-23m76f6e85ce3d7292b1982960eb22086ee03922166&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=Zta64bwn4nurTRpD4LY2OGr8KklkMRPn7Z_Qy0o4unU&m=SvyB_49EIFUT8-ZEVgIEYYjU6-zGTX4wb30kuNLUhkTSHYZK5-C0Gxr7uvefhtj4&s=e-fQOi0RzSZXxfSz37Bpz1sKtp7Yy0MWqonZnswK0RU&e=
>  

I've not gotten very far with this testing.  But I did find that regular
type 3 devices don't work with this change.  I used the patch below to get
this working.  Was there something I was missing to configure a non-DCD
device?

I don't particularly like adding another bool to this call stack.  Seems
like this calls for a flags field but I want to move on to DCD work so I
hacked this in.

I am working on the DCD FM-API commands here -
https://gitlab.com/sheshas/qemu-fmapi/-/tree/cxl-2023-05-25
-Shesha

Ira

commit ed27935044dcbd2c6ba71f8411b218621f3f4167
Author: Ira Weiny <ira.weiny@intel.com>
Date:   Fri Jun 9 13:56:33 2023 -0700

    hw/mem/cxl_type3: Exclude DCD from CEL when type3 is not DCD
    
    Per CXL 3.0 9.13.3 Dynamic Capacity Device (DCD) when the type 3 memory
    device does not have DCD support the CEL should not include DCD
    configuration commands.
    
    If the number of DC regions supported is 0 skip the DCD commands in the
    CEL.
    
    Applies on top of Fan Ni's work here:
    
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_moking_qemu-2Ddcd-2Dpreview-2Dlatest_tree_dcd-2Dpreview&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=Zta64bwn4nurTRpD4LY2OGr8KklkMRPn7Z_Qy0o4unU&m=SvyB_49EIFUT8-ZEVgIEYYjU6-zGTX4wb30kuNLUhkTSHYZK5-C0Gxr7uvefhtj4&s=MFD7qlSaTuy-w6aDmavIMbSP_aeaqZmSML7IVOX5jLs&e=
 
    
    Not-yet-Signed-off-by: Ira Weiny <ira.weiny@intel.com>

diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index a4a2c6a80004..262e35935563 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -288,7 +288,7 @@ static void mailbox_reg_init_common(CXLDeviceState 
*cxl_dstate)
 
 static void memdev_reg_init_common(CXLDeviceState *cxl_dstate) { }
 
-void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
+void cxl_device_register_init_common(CXLDeviceState *cxl_dstate, bool is_dcd)
 {
     uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
     const int cap_count = 3;
@@ -307,7 +307,7 @@ void cxl_device_register_init_common(CXLDeviceState 
*cxl_dstate)
     cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000, 1);
     memdev_reg_init_common(cxl_dstate);
 
-    cxl_initialize_mailbox(cxl_dstate, false);
+    cxl_initialize_mailbox(cxl_dstate, false, is_dcd);
 }
 
 void cxl_device_register_init_swcci(CXLDeviceState *cxl_dstate)
@@ -329,7 +329,7 @@ void cxl_device_register_init_swcci(CXLDeviceState 
*cxl_dstate)
     cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000, 1);
     memdev_reg_init_common(cxl_dstate);
 
-    cxl_initialize_mailbox(cxl_dstate, true);
+    cxl_initialize_mailbox(cxl_dstate, true, false);
 }
 
 uint64_t cxl_device_get_timestamp(CXLDeviceState *cxl_dstate)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 93b26e717c94..80e9cb9a8f04 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -1526,7 +1526,8 @@ static void bg_timercb(void *opaque)
     }
 }
 
-void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci)
+void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci,
+                            bool is_dcd)
 {
     if (!switch_cci) {
         cxl_dstate->cxl_cmd_set = cxl_cmd_set;
@@ -1534,6 +1535,9 @@ void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, 
bool switch_cci)
         cxl_dstate->cxl_cmd_set = cxl_cmd_set_sw;
     }
     for (int set = 0; set < 256; set++) {
+        if (!is_dcd && set == DCD_CONFIG) {
+            continue;
+        }
         for (int cmd = 0; cmd < 256; cmd++) {
             if (cxl_dstate->cxl_cmd_set[set][cmd].handler) {
                 struct cxl_cmd *c = &cxl_dstate->cxl_cmd_set[set][cmd];
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 329e8b5915b3..e6e6e125990c 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1276,9 +1276,11 @@ static void ct3d_reset(DeviceState *dev)
     CXLType3Dev *ct3d = CXL_TYPE3(dev);
     uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers;
     uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask;
+    bool is_dcd;
 
     cxl_component_register_init_common(reg_state, write_msk, 
CXL2_TYPE3_DEVICE);
-    cxl_device_register_init_common(&ct3d->cxl_dstate);
+    is_dcd = (ct3d->dc.num_regions != 0);
+    cxl_device_register_init_common(&ct3d->cxl_dstate, is_dcd);
 }
 
 static Property ct3_props[] = {
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 1ccddcca7d0d..4621bba4f533 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -233,7 +233,7 @@ typedef struct cxl_device_state {
 void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev);
 
 /* Set up default values for the register block */
-void cxl_device_register_init_common(CXLDeviceState *dev);
+void cxl_device_register_init_common(CXLDeviceState *dev, bool is_dcd);
 void cxl_device_register_init_swcci(CXLDeviceState *dev);
 
 /*
@@ -280,7 +280,7 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
                                       CXL_DEVICE_CAP_HDR1_OFFSET +
                                           CXL_DEVICE_CAP_REG_SIZE * 2)
 
-void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci);
+void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci, bool 
is_dcd);
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
 
 #define cxl_device_cap_init(dstate, reg, cap_id, ver)                      \