Re: Bitlocker-tripping hardware change with 5.2.0?

[Michael Weiser]

Hi Stefan,

On Sat, Dec 19, 2020 at 11:02:29PM -0500, Stefan Berger wrote:

> The whole purpose of measured/trusted boot is to reflect some known
> measurement values of a known BIOS in the TPM PCRs. Unfortunately this bites
> with sealing to those values and the rather fast development of QEMU.
> Updates of QEMU on the host platform then become a recovery event inside the
> VM, which is unfortunate, but this is when measured/trusted boot actually
> 'does its job'.

Thank you! This was a cruicial hint at exactly the right time: I was at
a point of severe head-scratching where I had gotten git bisect to work
but it was telling me that stock v5.1.0 compiled from git source also
showed the problem while my system-installed version wasn't.

The difference indeed is in the firmware blobs used: Compiled from
source, qemu uses the firmware files in the pc-bios subdirectory of the
build directory while my system's qemu looks into /usr/share/qemu where
bios{,-256k}.bin and vgabios-qxl.bin are provided by a different package
(seabios).

With that knowledge, I was able to make the git-compiled v5.1.0 qemu
behave the same as the one installed in the system by providing -L
/usr/share/qemu to it.

qemu v5.2.0 compiled from git source also does not drop into BitLocker
recovery when pointed at the unchanged system-installed qemu-5.1.0
/usr/share/qemu directory. This negates the suspicion that some change
in qemu proper carries over into the VM.

strace shows the following files being opened by qemu when starting my
testing VM:

bios.bin
bios-256k.bin
efi-e1000e.rom
vgabios-qxl.bin
kvmvapic.bin

Since bios{,-256k}.bin and vgabios-qxl.bin are provided by the seabios
package and therefore (in my particular case) stable across an update
from qemu-5.1.0 to 5.2.0 and kvmvapic.bin is identical in 5.1.0 and
5.2.0, this left only efi-e1000e.rom as reason for Bitlocker recovery
upon upgrade.

And indeed, reverting efi-e1000e.rom to the 5.1.0 version after
upgrading my system qemu made the Bitlocker recovery go away.

Armed with that knowledge, I was able to provide qemu with a VM-specific
firmware path from within libvirtd xml following documentation at
https://libvirt.org/drvqemu.html#qemucommand:

--- win10-bitlocker.xml.bak     2020-12-20 13:28:09.361141043 +0100
+++ win10-bitlocker.xml 2020-12-20 13:29:22.098750216 +0100
@@ -5,7 +5,7 @@
 or other application using the libvirt API.
 -->

-<domain type='kvm'>
+<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
   <name>win10-bitlocker</name>
   <uuid>bf566263-35e3-4dba-af8c-8ca85dba6a85</uuid>
   <metadata>
@@ -141,4 +141,8 @@
       <address type='pci' domain='0x0000' bus='0x04' slot='0x00'
function='0x0'/>
     </memballoon>
   </devices>
+  <qemu:commandline>
+    <qemu:arg value='-L'/>
+    <qemu:arg 
value='/home/user/.config/libvirt/qemu/firmware/bf566263-35e3-4dba-af8c-8ca85dba6a85'/>
+  </qemu:commandline>
 </domain>

# ls -la 
/home/user/.config/libvirt/qemu/firmware/bf566263-35e3-4dba-af8c-8ca85dba6a85
total 680
drwxr-xr-x 2 user group   4096 Dec 20 13:26 .
drwxr-xr-x 3 user group   4096 Dec 20 13:25 ..
-rw-r--r-- 1 user group 262144 Dec 20 13:25 bios-256k.bin
-rw-r--r-- 1 user group 131072 Dec 20 13:25 bios.bin
-rw-r--r-- 1 user group 240128 Dec 20 13:25 efi-e1000e.rom
-rw-r--r-- 1 user group   9216 Dec 20 13:26 kvmvapic.bin
-rw-r--r-- 1 user group  38912 Dec 20 13:25 vgabios-qxl.bin

Assuming I keep the virtual hardware specification stable
also (e.g. pc-q35-5.1) and don't add/remove devices, this should work
almost indefinitly, shouldn't it?

> >     For giggles I did enter the recovery key of the testing VM when
> >     prompted. It did boot up and showed Bitlocker enabled. After rebooting
> >     it prompted for the recovery key again. I entered it again, it booted
> >     again and I turned off Bitlocker (decrypted the disk). After
> >     re-enabling
> >     Bitlocker (re-encrypting the disk) and rebooting again it now does not
> >     prompt for the recovery key again.
> Does Bitlocker not allow you to accept a configuration change (= PCR value
> change) and it internally (presumably) then seals the secreted against those
> new values so upon the next reboot it works again? Do you really need to go
> through the whole re-encryption process? It sounds like unattractive even
> for a physical machine firmware update...

Indeed, upon retesting multiple times it seems that entering the
recovery key does actually also accept the firmware change. So there
must have been some user error on my part on the first try.

On closer look there is also the option of suspending Bitlocker
protection for a number of boots[1]. The data stays encrypted but
encryption keys are made available to the bootloader in the clear. Upon
resumption, Bitlocker silently re-measures the system and thus accepts
changes. When using the Control Panel applet[2], there's no option to
enter the number of boots and it's reactivated immediately after the
next boot.

I did a quick test of that: A change is accepted on the next boot and
recovery stays silent on following boots but an additional change at a
later time drops me back into recovery again. So it seems to work as
advertised.

So, suspending Bitlocker in every "bitlockered" VM before a qemu
upgrade, the user should never see any recovery screen. I would be
certain to forget about that every second time and end up needing to
downgrade qemu (and in my case possibly also the seabios package) again.

[1] 
https://docs.microsoft.com/en-us/powershell/module/bitlocker/suspend-bitlocker
[2] 
https://www.top-password.com/blog/wp-content/uploads/2019/02/suspend-bitlocker-protection.png

@Marc-Andre: Re: git bisect: It seems "git bisect v5.2.0 v5.1.0" does
something different than "git checkout v5.2.0 && git bisect start && git
bisect bad && git bisect good v5.1.0". With the latter command sequence
(and switching back to an out-of-source build), git bisect worked just
fine, including parallel build. I'm not too happy with that explanation
myself and would rather write this off to some additional layer 8
problem in the process.
-- 
Thanks,
Michael