On Sat, 18 May 2024, Andrew Randrianasulu wrote:
> On Sat, May 18, 2024 at 11:14 PM BALATON Zoltan <balaton@eik.bme.hu> wrote:
>
>> On Sat, 18 May 2024, Andrew Randrianasulu wrote:
>>> On Sat, May 18, 2024 at 10:24 PM BALATON Zoltan <balaton@eik.bme.hu>
>> wrote:
>>>
>>>> On Sat, 18 May 2024, Andrew Randrianasulu wrote:
>>>>> Using attached patch I get this new dmesg:
>>>>>
>>>>> Quiescing Open Firmware ...
>>>>>>> of_client_interface: quiesce
>>>>>>> of_client_interface return:
>>>>> Booting Linux via __start() @ 0x01000000 ...
>>>>> Hello World !
>>>>> [ 0.000000] Total memory = 512MB; using 1024kB for hash table
>>>>> [ 0.000000] Activating Kernel Userspace Execution Prevention
>>>>> [ 0.000000] Activating Kernel Userspace Access Protection
>>>>> [ 0.000000] Linux version 5.13.12_1 (voidlinux@voidlinux) (gcc
>> (GCC)
>>>> 10.2.1 20201203, GNU ld (GNU Binutils) 2.35.1) #1 SMP Thu Aug 19
>> 14:12:26
>>>> UTC 2021
>>>>> [ 0.000000] ioremap() called early from
>> pmac_feature_init+0xd4/0xad4.
>>>> Use early_ioremap() instead
>>>>> [ 0.000000] Found UniNorth memory controller & host bridge @
>>>> 0xf8000000 revision: 0x07
>>>>> [ 0.000000] Mapped at 0xf73c0000
>>>>> [ 0.000000] ioremap() called early from probe_one_macio+0x17c/0x2b4.
>>>> Use early_ioremap() instead
>>>>> [ 0.000000] Found a Keylargo mac-io controller, rev: 0, mapped at
>>>> 0x(ptrval)
>>>>> [ 0.000000] PowerMac motherboard: PowerMac G4 AGP Graphics
>>>>> [ 0.000000] ioremap() called early from udbg_scc_init+0x1e4/0x3f8.
>>>> Use early_ioremap() instead
>>>>> [ 0.000000] boot stdout isn't a display !
>>>>> [ 0.000000] ioremap() called early from find_via_cuda+0xb4/0x404.
>> Use
>>>> early_ioremap() instead
>>>>> [ 0.000000] Using PowerMac machine description
>>>>> [ 0.000000] printk: bootconsole [udbg0] enabled
>>>>> [ 0.000000] CPU maps initialized for 1 thread per core
>>>>> [ 0.000000] -----------------------------------------------------
>>>>> [ 0.000000] phys_mem_size = 0x20000000
>>>>> [ 0.000000] dcache_bsize = 0x20
>>>>> [ 0.000000] icache_bsize = 0x20
>>>>> [ 0.000000] cpu_features = 0x000000000501a00a
>>>>> [ 0.000000] possible = 0x00000000277de14a
>>>>> [ 0.000000] always = 0x0000000001000000
>>>>> [ 0.000000] cpu_user_features = 0x9c000001 0x00000000
>>>>> [ 0.000000] mmu_features = 0x00000001
>>>>> [ 0.000000] Hash_size = 0x100000
>>>>> [ 0.000000] Hash_mask = 0x3fff
>>>>> [ 0.000000] -----------------------------------------------------
>>>>> [ 0.000000] ioremap() called early from pmac_setup_arch+0x118/0x290.
>>>> Use early_ioremap() instead
>>>>> [ 0.000000] ioremap() called early from pmac_nvram_init+0x150/0x53c.
>>>> Use early_ioremap() instead
>>>>> [ 0.000000] nvram: Checking bank 0...
>>>>> [ 0.000000] Invalid signature
>>>>> [ 0.000000] Invalid checksum
>>>>> [ 0.000000] nvram: gen0=0, gen1=0
>>>>> [ 0.000000] nvram: Active bank is: 0
>>>>> [ 0.000000] nvram: OF partition at 0xffffffff
>>>>> [ 0.000000] nvram: XP partition at 0xffffffff
>>>>> [ 0.000000] nvram: NR partition at 0xffffffff
>>>>> [ 0.000000] Zone ranges:
>>>>> [ 0.000000] DMA [mem 0x0000000000000000-0x000000001fffffff]
>>>>> [ 0.000000] Normal empty
>>>>> [ 0.000000] HighMem empty
>>>>> [ 0.000000] Movable zone start for each node
>>>>> [ 0.000000] Early memory node ranges
>>>>> [ 0.000000] node 0: [mem 0x0000000000000000-0x000000001fffffff]
>>>>> [ 0.000000] Initmem setup node 0 [mem
>>>> 0x0000000000000000-0x000000001fffffff]
>>>>> [ 0.000000] percpu: Embedded 22 pages/cpu s59884 r8192 d22036 u90112
>>>>> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages:
>>>> 129920
>>>>> [ 0.000000] Kernel command line: console=ttyPZ0
>>>>> [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144
>>>> bytes, linear)
>>>>> [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072
>>>> bytes, linear)
>>>>> [ 0.000000] mem auto-init: stack:off, heap alloc:on, heap free:off
>>>>> [ 0.000000] Kernel virtual memory layout:
>>>>> [ 0.000000] * 0xf7bdf000..0xfffff000 : fixmap
>>>>> [ 0.000000] * 0xf7400000..0xf7800000 : highmem PTEs
>>>>> [ 0.000000] * 0xf7338000..0xf7400000 : early ioremap
>>>>> [ 0.000000] * 0xe1000000..0xf7338000 : vmalloc & ioremap
>>>>> [ 0.000000] Memory: 500988K/524288K available (10912K kernel code,
>>>> 1444K rwdata, 2512K rodata, 1420K init, 516K bss, 23300K reserved, 0K
>>>> cma-reserved, 0K highmem)
>>>>> [ 0.000000] random: get_random_u32 called from
>>>> __kmem_cache_create+0x38/0x57c with crng_init=0
>>>>> [ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4,
>> Nodes=1
>>>>> [ 0.000000] ftrace: allocating 29527 entries in 87 pages
>>>>> [ 0.000000] ftrace: allocated 87 pages with 5 groups
>>>>> [ 0.000000] rcu: Hierarchical RCU implementation.
>>>>> [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=2048 to
>>>> nr_cpu_ids=4.
>>>>> [ 0.000000] Trampoline variant of Tasks RCU enabled.
>>>>> [ 0.000000] Rude variant of Tasks RCU enabled.
>>>>> [ 0.000000] Tracing variant of Tasks RCU enabled.
>>>>> [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay
>>>> is 25 jiffies.
>>>>> [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16,
>>>> nr_cpu_ids=4
>>>>> [ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16
>>>>> [ 0.000000] mpic: Setting up MPIC " MPIC 1 " version 1.2 at
>>>> 80040000, max 4 CPUs
>>>>> [ 0.000000] mpic: ISU size: 64, shift: 6, mask: 3f
>>>>> [ 0.000000] mpic: Initializing for 64 sources
>>>>> [ 0.000000] GMT Delta read from XPRAM: 0 minutes, DST: on
>>>>> [ 0.001114] clocksource: timebase: mask: 0xffffffffffffffff
>>>> max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns
>>>>> [ 0.002661] clocksource: timebase mult[28000000] shift[24]
>> registered
>>>>> [ 0.022081] Console: colour dummy device 80x25
>>>>> [ 0.059474] printk: console [ttyPZ0] enabled
>>>>> [ 0.059474] printk: console [ttyPZ0] enabled
>>>>> [ 0.060950] printk: bootconsole [udbg0] disabled
>>>>> [ 0.060950] printk: bootconsole [udbg0] disabled
>>>>> [ 0.063583] pid_max: default: 32768 minimum: 301
>>>>> [ 0.066030] LSM: Security Framework initializing
>>>>> [ 0.068483] Yama: becoming mindful.
>>>>> [ 0.075446] AppArmor: AppArmor initialized
>>>>> [ 0.076811] Mount-cache hash table entries: 1024 (order: 0, 4096
>>>> bytes, linear)
>>>>> [ 0.077566] Mountpoint-cache hash table entries: 1024 (order: 0,
>> 4096
>>>> bytes, linear)
>>>>> smp_core99_probe
>>>>> [ 0.140858] PowerMac SMP probe found 4 cpus
>>>>
>>>> You're still using too many CPUs. Try to get close to real hardware and
>>>> only try 2 CPUs at first so we don't have to care about things that
>> don't
>>>> happen on real machine.
>>>>
>>>
>>> yeah, but linux define all four :}
>>>
>>> core99_reset_cpu(struct device_node *node, long param, long value)
>>>
>>> const int dflt_reset_lines[] = { KL_GPIO_RESET_CPU0,
>>> KL_GPIO_RESET_CPU1,
>>> KL_GPIO_RESET_CPU2,
>>> KL_GPIO_RESET_CPU3 };
>>>
>>> and those defined in keylargo.h as
>>>
>>> #define KEYLARGO_GPIO_EXTINT_0 0x58
>>>
>>> #define KL_GPIO_RESET_CPU0 (KEYLARGO_GPIO_EXTINT_0+0x03)
>>> #define KL_GPIO_RESET_CPU1 (KEYLARGO_GPIO_EXTINT_0+0x04)
>>> #define KL_GPIO_RESET_CPU2 (KEYLARGO_GPIO_EXTINT_0+0x0f)
>>> #define KL_GPIO_RESET_CPU3 (KEYLARGO_GPIO_EXTINT_0+0x10)
>>
>> Still it may be simpler to deal with only 1 additional CPU first just to
>> reduce complexity. So CPU reset seems to be done by core99_reset_cpu()
>> which will:
>>
>> 1. Look for something in open firmware device tree - is that there with
>> OpenBIOS, is it needed? Could be found by comparing to device tree from
>> real machine but you'd need to find a PowerMac3,3 dual CPU dump as
>> PowerMac3,1 only was single CPU. Otherwise it defaults to the above
>> defines.
>>
>
>
> const u32 *rst = of_get_property(np, "soft-reset", NULL); ?
>
> I can't see it in openfirmware. Should it be set for all cpus ?
No idea. Try to find device tree dump from real machine and check there.
not exactly of dump but dmesg from real dual g4
>>
>> 2. Poke mac-io to lower the reset line as far as I understand. Is this
>> emulated in QEMU and connected to reset the CPUs? The mac_newworld.c seems
>> to connect CPU reset to openpic not macio/keylargo or is that the same
>> device? Are these KEYLARGO_GPIO_EXTINT_0 registers emulated correctly?
>>
>>>>
>>>>> [ 0.142883] Processor timebase sync using GPIO 0x73
>>>>> [ 0.143474] mpic: requesting IPIs...
>>>>> [ 0.148795] CPU0: L2CR is 0
>>>>> [ 0.164657] rcu: Hierarchical SRCU implementation.
>>>>> [ 0.186502] smp: Bringing up secondary CPUs ...
>>>>> smp_core99_kick_cpu
>>>>> smp_core99_kick_cpu done
>>>>> [ 5.227920] Processor 1 is stuck.
>>>>> smp_core99_kick_cpu
>>>>> smp_core99_kick_cpu done
>>>>> [ 10.278140] Processor 2 is stuck.
>>>>> smp_core99_kick_cpu
>>>>> smp_core99_kick_cpu done
>>>>> [ 15.309326] Processor 3 is stuck.
>>>>
>>>> I guess we need to implement the thing that resets the additional CPUs
>> or
>>>> if that already happens then something else is wrong. I have no idea how
>>>> that works on real machine or what's in Linux or QEMU but the way to
>> find
>>>> out is ro check the Linux code to see what does it poke to reset the
>> other
>>>> CPU then check if that's handled in QEMU correctly and if not fix that.
>>>> (After that it will likely still hang because of IRQ routing but maybe
>>>> should see the CPUs at least.)
>>>>
>>>>> [ 15.310457] smp: Brought up 1 node, 1 CPU
>>>>> [ 15.336741] devtmpfs: initialized
>>>>> [ 15.358958] Found UniNorth PCI host bridge at 0x00000000f2000000.
>>>> Firmware bus number: 0->0
>>>>> [ 15.359907] PCI host bridge /pci@f2000000 (primary) ranges:
>>>>> [ 15.362134] IO 0x00000000f2000000..0x00000000f27fffff ->
>>>> 0x0000000000000000
>>>>> [ 15.363327] MEM 0x0000000080000000..0x000000008fffffff ->
>>>> 0x0000000080000000
>>>>> [ 15.366636] clocksource: jiffies: mask: 0xffffffff max_cycles:
>>>> 0xffffffff, max_idle_ns: 7645041785100000 ns
>>>>> [ 15.367915] futex hash table entries: 1024 (order: 3, 32768 bytes,
>>>> linear)
>>>>> [ 15.407396] NET: Registered protocol family 16
>>>>> [ 15.416447] audit: initializing netlink subsys (disabled)
>>>>> [ 15.423487] audit: type=2000 audit(1716054549.076:1):
>>>> state=initialized audit_enabled=0 res=1
>>>>> [ 15.433124] thermal_sys: Registered thermal governor 'fair_share'
>>>>>
>>>>> [ 15.433296] thermal_sys: Registered thermal governor 'step_wise'
>>>>> [ 15.444053] PCI: Probing PCI hardware
>>>>> [ 15.454648] PCI host bridge to bus 0000:00
>>>>> [ 15.455908] pci_bus 0000:00: root bus resource [io 0x0000-0x7fffff]
>>>>> [ 15.457308] pci_bus 0000:00: root bus resource [mem
>>>> 0x80000000-0x8fffffff]
>>>>> [ 15.458521] pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>> [ 15.459767] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to
>>>> ff
>>>>> [ 15.464530] pci 0000:00:0b.0: [106b:001f] type 00 class 0x060000
>>>>> [ 15.473861] pci 0000:00:0c.0: [106b:0022] type 00 class 0xff0000
>>>>> [ 15.475845] pci 0000:00:0c.0: reg 0x10: [mem 0x80000000-0x8007ffff]
>>>>> [ 15.481484] pci 0000:00:0d.0: [106b:003f] type 00 class 0x0c0310
>>>>> [ 15.482883] pci 0000:00:0d.0: reg 0x10: [mem 0x80080000-0x800800ff]
>>>>> [ 15.488606] pci 0000:00:0e.0: [1234:1111] type 00 class 0x030000
>>>>> [ 15.490521] pci 0000:00:0e.0: reg 0x10: [mem 0x81000000-0x81ffffff
>>>> pref]
>>>>> [ 15.494643] pci 0000:00:0e.0: reg 0x18: [mem 0x82000000-0x82000fff]
>>>>> [ 15.501794] pci 0000:00:0e.0: reg 0x30: [mem 0x82010000-0x8201ffff
>>>> pref]
>>>>> [ 15.503465] pci 0000:00:0f.0: [106b:0021] type 00 class 0x020000
>>>>> [ 15.504560] pci 0000:00:0f.0: reg 0x10: [mem 0x82200000-0x823fffff]
>>>>> [ 15.511499] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to
>>>> 00
>>>>> [ 15.520044] pci_bus 0000:00: resource 4 [io 0x0000-0x7fffff]
>>>>> [ 15.520695] pci_bus 0000:00: resource 5 [mem 0x80000000-0x8fffffff]
>>>>> [ 15.541800] Kprobes globally optimized
>>>>> [ 20.205864] iommu: Default domain type: Translated
>>>>> [ 20.214125] pci 0000:00:0e.0: vgaarb: setting as boot VGA device
>>>>> [ 20.215070] pci 0000:00:0e.0: vgaarb: VGA device added:
>>>> decodes=io+mem,owns=io+mem,locks=none
>>>>> [ 20.216002] pci 0000:00:0e.0: vgaarb: bridge control possible
>>>>> [ 20.216606] vgaarb: loaded
>>>>> [ 20.220636] SCSI subsystem initialized
>>>>> [ 20.253578] clocksource: Switched to clocksource timebase
>>>>> [ 20.452951] VFS: Disk quotas dquot_6.6.0
>>>>> [ 20.453727] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096
>>>> bytes)
>>>>> [ 20.460869] AppArmor: AppArmor Filesystem Enabled
>>>>> [ 20.495964] NET: Registered protocol family 2
>>>>> [ 20.498712] IP idents hash table entries: 8192 (order: 4, 65536
>>>> bytes, linear)
>>>>> [ 20.521739] tcp_listen_portaddr_hash hash table entries: 512 (order:
>>>> 0, 6144 bytes, linear)
>>>>> [ 20.522799] TCP established hash table entries: 4096 (order: 2,
>> 16384
>>>> bytes, linear)
>>>>> [ 20.523935] TCP bind hash table entries: 4096 (order: 3, 32768
>> bytes,
>>>> linear)
>>>>> [ 20.524986] TCP: Hash tables configured (established 4096 bind 4096)
>>>>> [ 20.529295] MPTCP token hash table entries: 512 (order: 1, 8192
>>>> bytes, linear)
>>>>> [ 20.530582] UDP hash table entries: 256 (order: 1, 8192 bytes,
>> linear)
>>>>> [ 20.531628] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes,
>>>> linear)
>>>>> [ 20.536784] NET: Registered protocol family 1
>>>>> [ 20.537953] NET: Registered protocol family 44
>>>>> [ 20.542641] PCI: CLS 0 bytes, default 32
>>>>> [ 20.550784] Thermal assist unit using workqueue, shrink_timer: 2000
>> ms
>>>>> [ 20.567668] Initialise system trusted keyrings
>>>>> [ 20.572356] workingset: timestamp_bits=14 max_order=17
>> bucket_order=3
>>>>> [ 20.594107] zbud: loaded
>>>>> [ 20.601453] Key type asymmetric registered
>>>>> [ 20.602101] Asymmetric key parser 'x509' registered
>>>>> [ 20.602938] Block layer SCSI generic (bsg) driver version 0.4 loaded
>>>> (major 250)
>>>>> [ 20.604717] io scheduler mq-deadline registered
>>>>> [ 20.605353] io scheduler kyber registered
>>>>> [ 20.606241] io scheduler bfq registered
>>>>> [ 20.616132] Using unsupported 800x600 (null) at 81000000, depth=32,
>>>> pitch=3200
>>>>> [ 20.643222] Console: switching to colour frame buffer device 100x37
>>>>> [ 20.652163] fb0: Open Firmware frame buffer device on /pci@f2000000
>>>> /QEMU,VGA@e
>>>>> [ 20.655049] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
>>>>> [ 20.665430] pmac_zilog: 0.6 (Benjamin Herrenschmidt <
>>>> benh@kernel.crashing.org>)
>>>>> [ 20.666517] Serial: MPC52xx PSC UART driver
>>>>> [ 20.667710] Non-volatile memory driver v1.3
>>>>> [ 20.668168] Linux agpgart interface v0.103
>>>>> [ 20.672974] MacIO PCI driver attached to Keylargo chipset
>>>>> [ 20.681690] 0.00013020:ch-a: ttyPZ0 at MMIO 0x80013020 (irq = 37,
>>>> base_baud = 230400) is a Z85c30 ESCC - Serial port
>>>>> [ 20.685655] 0.00013000:ch-b: ttyPZ1 at MMIO 0x80013000 (irq = 36,
>>>> base_baud = 230400) is a Z85c30 ESCC - Serial port
>>>>> [ 20.691584] Macintosh Cuda and Egret driver.
>>>>
>>>> This reminds me that mac99 is not actually PowerMac3.1 that it claims to
>>>> be so make sure to always use -M mac99,via=pmu instead which is what
>>>> actually like a PowerMac. Also I don't know if PMU plays a role in SMP
>> or
>>>> not but let's try to match real hardware here.
>>>>
>>>> Here's the patch inline so it's easier to reply:
>>>>
>>>> --- /home/guest/botva/src/src/qemu/hw/ppc/mac_newworld.c
>> 2024-03-27
>>>> 06:44:12.068828324 +0300
>>>> +++ hw/ppc/mac_newworld.c 2024-05-18 20:48:32.709184875 +0300
>>>> @@ -128,6 +128,15 @@
>>>> cpu->env.nip = PROM_BASE + 0x100;
>>>> }
>>>>
>>>> +
>>>> +static void ppc_core99_reset_sec(void *opaque)
>>>> +{
>>>> + PowerPCCPU *cpu = opaque;
>>>> + CPUState *cs = CPU(cpu);
>>>> + cpu_reset(cs);
>>>> + cs->exception_index = EXCP_HLT;
>>>> +}
>>>> +
>>>> /* PowerPC Mac99 hardware initialisation */
>>>> static void ppc_core99_init(MachineState *machine)
>>>> {
>>>> @@ -160,10 +169,27 @@
>>>> for (i = 0; i < machine->smp.cpus; i++) {
>>>> cpu = POWERPC_CPU(cpu_create(machine->cpu_type));
>>>> env = &cpu->env;
>>>> + CPUState *cs;
>>>> + cs = CPU(cpu);
>>>>
>>>> ! I think ou can cast cpu to OBJECT(cpu) then no cs is needed.
>>>>
>>>> +
>>>> + /*
>>>> + * Secondary CPU starts in halted state for now. Needs to change
>>>> + * when implementing non-kernel boot.
>>>> + */
>>>> + object_property_set_bool(OBJECT(cs), "start-powered-off", i
>> != 0,
>>>> + &error_abort);
>>>>
>>>> ! Now I'm not sure the ppc_core99_reset_sec() is really needed or this
>>>> ! property is enough. e500 seems to do both and some others too but
>> these
>>>> ! seem to do tha same thing and halt CPU on reset. Whay do we have two
>>>> ! ways for this and is both needed?
>>>>
>>>> + //qdev_realize_and_unref(DEVICE(cs), NULL, &error_fatal);
>>>>
>>>> ! If you remove raliziation of CPU object how does it even work? I think
>>>> ! this line is still needed,
>>>>
>>>
>>>
>>> with it machine not even start and assert somewhere ...
>>
>> Where and why? What's the assert? Without realizing the object might not
>> completely initialised so I'm surprised it works at all. Instead of
>> removing realize we should find out why it asserts as that might mean
>> there's some problem that needs to be fixed.
>>
>
> qemu-system-ppc: ../hw/core/qdev.c:282: qdev_realize: Assertion
> `!dev->realized && !dev->parent_bus' failed
You're probably calling realize multiple times on the same object. Indeed
cpu_create realizes the CPU so you need to open code it (see
hw/core/cpu-common.c) and replace cpu_create with object_new so you can
set property and then call realize and unref. That should fix the assert
but don't know if this is needed or if both reset_sec and
start-powered-off property are needed or one of them is enough.
Regards,
BALATON Zoltan