Re: [PATCH 2/2] qemu-iotests: Test convert to qcow2 compressed to NBD

[Nir Soffer]

On Mon, Jul 27, 2020 at 1:05 PM Max Reitz <mreitz@redhat.com> wrote:
>
> On 26.07.20 17:25, Nir Soffer wrote:
> > Add test for "qemu-img convert -O qcow2 -c" to NBD target. The use case
> > is writing compressed disk content to OVA archive.
> >
> > Signed-off-by: Nir Soffer <nsoffer@redhat.com>
> > ---
> >  tests/qemu-iotests/302     | 83 ++++++++++++++++++++++++++++++++++++++
> >  tests/qemu-iotests/302.out | 27 +++++++++++++
> >  tests/qemu-iotests/group   |  1 +
> >  3 files changed, 111 insertions(+)
> >  create mode 100755 tests/qemu-iotests/302
> >  create mode 100644 tests/qemu-iotests/302.out
> >
> > diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
> > new file mode 100755
> > index 0000000000..cefde1f7cf
> > --- /dev/null
> > +++ b/tests/qemu-iotests/302
> > @@ -0,0 +1,83 @@
> > +#!/usr/bin/env python3
> > +#
> > +# Tests conveting qcow2 compressed to NBD
>
> *converting
>
> > +#
> > +# Copyright (c) 2020 Nir Soffer <nirsof@gmail.com>
> > +#
> > +# This program is free software; you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License as published by
> > +# the Free Software Foundation; either version 2 of the License, or
> > +# (at your option) any later version.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> > +#
> > +# owner=nirsof@gmail.com
> > +
> > +import json
> > +import iotests
> > +
> > +from iotests import (
> > +    file_path,
> > +    qemu_img,
> > +    qemu_img_create,
> > +    qemu_img_log,
> > +    qemu_img_pipe,
> > +    qemu_io,
> > +    qemu_nbd,
> > +)
> > +
> > +iotests.script_initialize(supported_fmts=["qcow2"])
> > +
> > +# Create source disk, format does not matter.
> > +src_disk = file_path("disk.img")
> > +qemu_img_create("-f", "raw", src_disk, "10m")
>
> If the format doesn’t matter, why not just use qcow2 and so put
> iotests.imgfmt here?  (And everywhere else where you now have -f raw.)

I tried to use the simplest setup that is less likely to break, but
thinking about
CI environments with strange storage, maybe using qcow2 source disk will be
more robust.

> > +qemu_io("-f", "raw", "-c", "write 1m 64K", src_disk)
>
> (Except I think qemu_io already has -f qcow2 in its arguments by
> default, so specifying the format wouldn’t even be necessary here.)
>
> > +# The use case is writing qcow2 image directly into a tar file. Code to 
> > create
> > +# real tar file not included.
> > +#
> > +# offset    content
> > +# -------------------------------
> > +#      0    first memebr header
>
> *member
>
> > +#    512    first member data
> > +#   1024    second memeber header
>
> *member
>
> > +#   1536    second member data
> > +
> > +tar_file = file_path("test.tar")
> > +out = qemu_img_pipe("measure", "-O", "qcow2", "--output", "json", src_disk)
> > +measure = json.loads(out)
> > +qemu_img_create("-f", "raw", tar_file, str(measure["required"]))
>
> Should this be measure["required"] + 1536?
>
> > +
> > +nbd_sock = file_path("nbd-sock", base_dir=iotests.sock_dir)
> > +nbd_uri = "nbd+unix:///exp?socket=" + nbd_sock
> > +
> > +# Use raw format to allow creating qcow2 directy into tar file.
> > +qemu_nbd(
> > +    "--socket", nbd_sock,
> > +    "--persistent",
> > +    "--export-name", "exp",
> > +    "--format", "raw",
> > +    "--offset", "1536",
> > +    tar_file)
> > +
> > +iotests.log("=== Target image info ===")
> > +qemu_img_log("info", nbd_uri)
> > +
> > +# Write image into the tar file. In a real applicatio we would write a tar
>
> *application
>
> > +# entry after writing the image.
> > +qemu_img("convert", "-f", "raw", "-O", "qcow2", "-c", src_disk, nbd_uri)
> > +
> > +iotests.log("=== Converted image info ===")
> > +qemu_img_log("info", nbd_uri)
> > +
> > +iotests.log("=== Converted image check ===")
> > +qemu_img_log("check", nbd_uri)
> > +
> > +iotests.log("=== Comparing to source disk ===")
> > +qemu_img_log("compare", src_disk, nbd_uri)
> > diff --git a/tests/qemu-iotests/302.out b/tests/qemu-iotests/302.out
> > new file mode 100644
> > index 0000000000..babef3d574
> > --- /dev/null
> > +++ b/tests/qemu-iotests/302.out
> > @@ -0,0 +1,27 @@
> > +=== Target image info ===
> > +image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock
> > +file format: raw
> > +virtual size: 446 KiB (457216 bytes)
> > +disk size: unavailable
> > +
> > +=== Converted image info ===
> > +image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock
> > +file format: qcow2
> > +virtual size: 10 MiB (10485760 bytes)
> > +disk size: unavailable
> > +cluster_size: 65536
> > +Format specific information:
> > +    compat: 1.1
> > +    compression type: zlib
> > +    lazy refcounts: false
> > +    refcount bits: 16
> > +    corrupt: false
> > +
> > +=== Converted image check ===
> > +No errors were found on the image.
> > +1/160 = 0.62% allocated, 100.00% fragmented, 100.00% compressed clusters
> > +Image end offset: 393216
>
> I hope none of this is fs-dependant.  (I don’t think it is, but who
> knows.  I suppose we’ll find out.)

I don't think the check would be affected, but maybe the compare step
can use the strict
option if we change to qcow2 source disk.

>
> Max
>
> > +=== Comparing to source disk ===
> > +Images are identical.
> > +
> > diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
> > index 1d0252e1f0..1e1cb27bc8 100644
> > --- a/tests/qemu-iotests/group
> > +++ b/tests/qemu-iotests/group
> > @@ -308,3 +308,4 @@
> >  297 meta
> >  299 auto quick
> >  301 backing quick
> > +302 quick
> >
>
>