Re: [Qemu-devel] [PATCH V2] pci: fixes to allow booting from extra root pci buses.

[Laszlo Ersek]

On 06/11/15 19:46, Marcel Apfelbaum wrote:
> On 06/11/2015 07:54 PM, Kevin O'Connor wrote:
>> On Thu, Jun 11, 2015 at 05:36:06PM +0300, Marcel Apfelbaum wrote:
>>> On 06/11/2015 05:24 PM, Kevin O'Connor wrote:
>>>> On Thu, Jun 11, 2015 at 05:12:33PM +0300, Marcel Apfelbaum wrote:
>>>>> On 06/11/2015 04:58 PM, Kevin O'Connor wrote:
>>>>>> On Thu, Jun 11, 2015 at 04:37:08PM +0300, Marcel Apfelbaum wrote:
>>>>>>> The fixes solves the following issue:
>>>>>>> The PXB device exposes a new  pci root bridge with the
>>>>>>> fw path:  /address@hidden/..., in which 4 is the root bus number.
>>>>>>> Before this patch the fw path was wrongly computed:
>>>>>>>      /address@hidden/address@hidden/...
>>>>>>> Fix the above issues: Correct the bus number and remove the
>>>>>>> extra host bridge description.
>>>>>>
>>>>>> Why is that wrong?  The previous path looks correct to me.
>>>>> The prev path includes both the extra root bridge and *then* the
>>>>> usual host bridge.
>>>>>   /address@hidden/address@hidden/   ...
>>>>>      ^ new       ^ regular  ^ devices
>>>>>
>>>>> Since the new pci root bridge (and bus) is on "paralel" with the
>>>>> regular one.
>>>>> it is not correct to add it to the path.
>>>>>
>>>>> The architecture is:
>>>>>   /<host bridge>/devices...
>>>>>   /extra root bridge/devices...
>>>>>   /extra root bridge/devices...
>>>>> And not
>>>>> /extra root bridge//<host bridge>/devices
>>>>
>>>> Your patch changed both the "/extra root bridge/devices..." part and
>>>> the "@1" part.  The change of the "@1" in "/address@hidden/" is not
>>>> correct IMO.
>>> Why? @1 should be the unit address which is the text representation
>>> of the physical address, in our case the slot. Since the bus number
>>> in our case is 4, I think /address@hidden/ is the 'correct' address.
>>
>> On real machines, the firmware assigns the 4 - it's not a physical
>> address; it's a logical address (like all bus numbers in PCI).  The
>> firmware might assign a totally different number on the next boot.
> Now I am confused. Don't get me wrong, I am not an expert on fw, I hardly
> try to understand it.
> 
> I looked up a real hardware machine and it seemed to me that the extra
> pci root numbers
> are provided in the ACPI tables, meaning by the vendor, not the fw.
> In this case QEMU is the vendor, i440fx is the machine, right?
> 
> I am not aware that Seabios/OVMF are deciding the bus numbers for the
> *PCI roots*.
> They are doing it for the pci-2-pci bridges of course.
> I saw that Seabios is trying to "guess" the root-buses by going over all
> the 0-0xff range
> and probing all the slots, looking for devices. So it expects the hw to
> be hardwired regarding
> PCI root buses.

This is exactly how I understood it.

We're not interested in placing such bus numbers in device paths that
are assigned during PCI enumeration. (Like subordinate bus numbers.)
We're talking about the root bus numbers.

OVMF implements the same kind of probing that SeaBIOS does (based on
natural language description from Michael and Marcel, not on the actual
code). Devices on the root buses respond without any prior bus number
assignments. Therefore it makes sense to place those root bus numbers
into device paths.

The bus numbers assignable by the firmware come from the intervals
*between* the (fixed-in-hardware) root bus numbers. As I understand it,
for two adjacent root bus numbers R1 and R2, the both-sides-exclusive
interval (R1, R2) is available for secondary bus number assignment, to
non-root buses that are (recursively) behind PCI bridges that hang off
the R1 root bus (ie. the LHS of the interval).

We don't care about such firmware-assigned bus numbers at all, but R1,
R2 etc. must be communicated to the firmware *somehow* in order to
identify devices for booting.

Since R1, R2 etc are not *assigned* by the firmware, only detected (the
assignment happens in QEMU, and also by the hw vendor in case of
physical hardware), R1, R2 etc are permanent as long as the physical
configuration does not change. Hence they qualify for the physical
addressing nature of OFW device paths. We've just been looking for a
*syntax* to express them.

> Is my understanding incorrect?

FWIW I'm relieved that at least the two of us have been understanding
each other ;)

Laszlo