[Qemu-commits] [qemu/qemu] ee1e66: qemu-iotests: ignore leaks on failure paths in 026

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: ee1e66d9570a763a07662883093bb297ba22c5ee
      
https://github.com/qemu/qemu/commit/ee1e66d9570a763a07662883093bb297ba22c5ee
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/026
    M tests/qemu-iotests/026.out
    M tests/qemu-iotests/026.out.nocache
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: ignore leaks on failure paths in 026

Upcoming asynchronous handling of sub-parts of qcow2 requests will
change number of leaked clusters and even make it racy. As a
preparation, ignore leaks on failure parts in 026.

It's not trivial to just grep or substitute qemu-img output for such
thing. Instead do better: 3 is a error code of qemu-img check, if only
leaks are found. Catch this case and print success output.

Suggested-by: Anton Nefedov <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6e9b225f737d53afae8160d73da72ee8f38eb1a6
      
https://github.com/qemu/qemu/commit/6e9b225f737d53afae8160d73da72ee8f38eb1a6
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/Makefile.objs
    A block/aio_task.c
    A include/block/aio_task.h

  Log Message:
  -----------
  block: introduce aio task pool

Common interface for aio task loops. To be used for improving
performance of synchronous io loops in qcow2, block-stream,
copy-on-read, and may be other places.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 88f468e54696cd9ffc28ecf96f18f04125402541
      
https://github.com/qemu/qemu/commit/88f468e54696cd9ffc28ecf96f18f04125402541
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  block/qcow2: refactor qcow2_co_preadv_part

Further patch will run partial requests of iterations of
qcow2_co_preadv in parallel for performance reasons. To prepare for
this, separate part which may be parallelized into separate function
(qcow2_co_preadv_task).

While being here, also separate encrypted clusters reading to own
function, like it is done for compressed reading.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6aa7a2631b88412cc85aeb67c1e188517c54fa5e
      
https://github.com/qemu/qemu/commit/6aa7a2631b88412cc85aeb67c1e188517c54fa5e
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  block/qcow2: refactor qcow2_co_pwritev_part

Similarly to previous commit, prepare for parallelizing write-loop
iterations.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: d710cf575ad5fb3ab329204620de45bfe50caa53
      
https://github.com/qemu/qemu/commit/d710cf575ad5fb3ab329204620de45bfe50caa53
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/qcow2.c
    M block/qcow2.h
    M block/trace-events

  Log Message:
  -----------
  block/qcow2: introduce parallel subrequest handling in read and write

It improves performance for fragmented qcow2 images.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 981fb5810aa3f68797ee6e261db338bd78857614
      
https://github.com/qemu/qemu/commit/981fb5810aa3f68797ee6e261db338bd78857614
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: fix max_transfer handling for copy_range

Of course, QEMU_ALIGN_UP is a typo, it should be QEMU_ALIGN_DOWN, as we
are trying to find aligned size which satisfy both source and target.
Also, don't ignore too small max_transfer. In this case seems safer to
disable copy_range.

Fixes: 9ded4a0114968e
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1048ddf0a32dcdaa952e581bd503d49adad527cc
      
https://github.com/qemu/qemu/commit/1048ddf0a32dcdaa952e581bd503d49adad527cc
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: fix backup_cow_with_offload for last cluster

We shouldn't try to copy bytes beyond EOF. Fix it.

Fixes: 9ded4a0114968e
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0bd0c44372868432c6bde5b0022974f65241f105
      
https://github.com/qemu/qemu/commit/0bd0c44372868432c6bde5b0022974f65241f105
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: split shareable copying part from backup_do_cow

Split copying logic which will be shared with backup-top filter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 372c67ea6146cfa4e886ee33191d48cb2df5e99c
      
https://github.com/qemu/qemu/commit/372c67ea6146cfa4e886ee33191d48cb2df5e99c
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: improve comment about image fleecing

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2c8074c453ff13a94bd08ec26061917670ec03be
      
https://github.com/qemu/qemu/commit/2c8074c453ff13a94bd08ec26061917670ec03be
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c
    M block/trace-events

  Log Message:
  -----------
  block/backup: introduce BlockCopyState

Split copying code part from backup to "block-copy", including separate
state structure and function renaming. This is needed to share it with
backup-top filter driver in further commits.

Notes:

1. As BlockCopyState keeps own BlockBackend objects, remaining
job->common.blk users only use it to get bs by blk_bs() call, so clear
job->commen.blk permissions set in block_job_create and add
job->source_bs to be used instead of blk_bs(job->common.blk), to keep
it more clear which bs we use when introduce backup-top filter in
further commit.

2. Rename s/initializing_bitmap/skip_unallocated/ to sound a bit better
as interface to BlockCopyState

3. Split is not very clean: there left some duplicated fields, backup
code uses some BlockCopyState fields directly, let's postpone it for
further improvements and keep this comment simpler for review.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0e23e382b77baeff1f115ae06424b99d4449a594
      
https://github.com/qemu/qemu/commit/0e23e382b77baeff1f115ae06424b99d4449a594
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: fix block-comment style

We need to fix comment style around block-copy functions before further
moving them to separate file to satisfy checkpatch. But do more: fix
all comments style. Also, seems like doubled first asterisk is not
forbidden, but drop it too for consistency.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: beb5f5450d983bb4c560582efb5267813aef4065
      
https://github.com/qemu/qemu/commit/beb5f5450d983bb4c560582efb5267813aef4065
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/Makefile.objs
    M block/backup.c
    A block/block-copy.c
    M block/trace-events
    A include/block/block-copy.h

  Log Message:
  -----------
  block: move block_copy from block/backup.c to separate file

Split block_copy to separate file, to be cleanly shared with backup-top
filter driver in further commits.

It's a clean movement, the only change is drop "static" from interface
functions.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: d10529a2b7e6c44d5e4e278ca08fe4cdb357e25d
      
https://github.com/qemu/qemu/commit/d10529a2b7e6c44d5e4e278ca08fe4cdb357e25d
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: teach bdrv_debug_breakpoint skip filters with backing

Teach bdrv_debug_breakpoint and bdrv_debug_remove_breakpoint skip
filters with backing. This is needed to implement and use in backup job
it's own backup_top filter driver (like mirror already has one), and
without this improvement, breakpoint removal will fail at least in 55
iotest.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5c4343b89d4ca4383af0a41fd6de9db78bf0b1ac
      
https://github.com/qemu/qemu/commit/5c4343b89d4ca4383af0a41fd6de9db78bf0b1ac
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/124
    M tests/qemu-iotests/257
    M tests/qemu-iotests/257.out
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: prepare 124 and 257 bitmap querying for backup-top filter

After backup-top filter appearing it's not possible to see dirty
bitmaps in top node, so use node-name instead.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: f1648454794c09f727f9b73d767a60caf94e87a6
      
https://github.com/qemu/qemu/commit/f1648454794c09f727f9b73d767a60caf94e87a6
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/257

  Log Message:
  -----------
  iotests: 257: drop unused Drive.device field

After previous commit Drive.device is actually unused. Drop it together
with .name property.  While being here reuse .node in qmp commands
instead of writing 'drive0' twice.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 15e4e6e6ad31e66f9a7ef558faf507810ba83636
      
https://github.com/qemu/qemu/commit/15e4e6e6ad31e66f9a7ef558faf507810ba83636
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/257
    M tests/qemu-iotests/257.out

  Log Message:
  -----------
  iotests: 257: drop device_add

SCSI devices are unused in test, drop them.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 329d27e38cdb450a29ce21acec724a9bfb69f13f
      
https://github.com/qemu/qemu/commit/329d27e38cdb450a29ce21acec724a9bfb69f13f
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M qapi/block-core.json

  Log Message:
  -----------
  qapi: group BlockDeviceStats fields

Make the stat fields definition slightly more readable.
Also reorder total_time_ns stats read-write-flush as done elsewhere.
Cosmetic change only.

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 159f85ddc81d56f9bd8c3822d5e7d59814c58e50
      
https://github.com/qemu/qemu/commit/159f85ddc81d56f9bd8c3822d5e7d59814c58e50
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/qapi.c
    M include/block/accounting.h
    M qapi/block-core.json
    M tests/qemu-iotests/227.out

  Log Message:
  -----------
  qapi: add unmap to BlockDeviceStats

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: f34444665481bf470c35182c2286de5eaf1465ce
      
https://github.com/qemu/qemu/commit/f34444665481bf470c35182c2286de5eaf1465ce
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/accounting.c
    M include/block/accounting.h

  Log Message:
  -----------
  block: add empty account cookie type

Each block_acct_done/failed call is designed to correspond to a
previous block_acct_start call, which initializes the stats cookie.
However sometimes it is not the case, e.g. some error paths might
report the same cookie twice because it is hard to accurately track if
the cookie was reported yet or not.

This patch cleans the cookie after report.
(Note: block_acct_failed/done without a previous block_acct_start at
all should be avoided. Uninitialized cookie might hold a garbage value
and there is still "< BLOCK_MAX_IOTYPE" assertion for that)

It will be particularly useful in ide code where it's hard to
keep track whether the request done its accounting or not: in the
following patch of the series, trim requests will do the accounting
separately.

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 99f18035b5e0d4f90a35e29022f1b4697fbe2835
      
https://github.com/qemu/qemu/commit/99f18035b5e0d4f90a35e29022f1b4697fbe2835
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M hw/ide/core.c

  Log Message:
  -----------
  ide: account UNMAP (TRIM) operations

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6d0680823c829b2d59f6b5488401b7c817d220b9
      
https://github.com/qemu/qemu/commit/6d0680823c829b2d59f6b5488401b7c817d220b9
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M hw/scsi/scsi-disk.c

  Log Message:
  -----------
  scsi: store unmap offset and nb_sectors in request struct

it allows to report it in the error handler

Signed-off-by: Anton Nefedov <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 90ebf8431e346638aa83ec8118f3540735cf4ecf
      
https://github.com/qemu/qemu/commit/90ebf8431e346638aa83ec8118f3540735cf4ecf
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M hw/scsi/scsi-disk.c

  Log Message:
  -----------
  scsi: move unmap error checking to the complete callback

This will help to account the operation in the following commit.

The difference is that we don't call scsi_disk_req_check_error() before
the 1st discard iteration anymore. That function also checks if
the request is cancelled, however it shouldn't get canceled until it
yields in blk_aio() functions anyway.
Same approach is already used for emulate_write_same.

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4989ef5781835bca2d02c621b52328ccb6253b61
      
https://github.com/qemu/qemu/commit/4989ef5781835bca2d02c621b52328ccb6253b61
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M hw/scsi/scsi-disk.c

  Log Message:
  -----------
  scsi: account unmap operations

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1c450366365a01c5c1c354af68b8aac5cf5f18c9
      
https://github.com/qemu/qemu/commit/1c450366365a01c5c1c354af68b8aac5cf5f18c9
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  file-posix: account discard operations

This will help to identify how many of the user-issued discard operations
(accounted on a device level) have actually suceeded down on the host file
(even though the numbers will not be exactly the same if non-raw format
driver is used (e.g. qcow2 sending metadata discards)).

Note that these numbers will not include discards triggered by
write-zeroes + MAY_UNMAP calls.

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: d924559953c0b679582548d8fd8af025667051e3
      
https://github.com/qemu/qemu/commit/d924559953c0b679582548d8fd8af025667051e3
  Author: Anton Nefedov <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block.c
    M block/file-posix.c
    M block/qapi.c
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json

  Log Message:
  -----------
  qapi: query-blockstat: add driver specific file-posix stats

A block driver can provide a callback to report driver-specific
statistics.

file-posix driver now reports discard statistics

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Acked-by: Markus Armbruster <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: e6e8db0337307eb600d0903adea44eaa7ff14cc3
      
https://github.com/qemu/qemu/commit/e6e8db0337307eb600d0903adea44eaa7ff14cc3
  Author: Max Reitz <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/125

  Log Message:
  -----------
  iotests: Fix 125 for growth_mode = metadata

If we use growth_mode = metadata, it is very much possible that the file
uses more disk space after we have written something to the added area.
We did indeed want to test for this case, but unfortunately we evidently
just copied the code from the "Test creation preallocation" section and
forgot to replace "$create_mode" by "$growth_mode".

We never noticed because we only read the first number from qemu-img
info's "disk size" output -- and that is effectively useless, because
qemu-img prints a human-readable value (which generally includes a
decimal point).  That will be fixed in the patch after the next one.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 285f595d74738f9f99c5b5b05005956e0d7c1b77
      
https://github.com/qemu/qemu/commit/285f595d74738f9f99c5b5b05005956e0d7c1b77
  Author: Max Reitz <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/125

  Log Message:
  -----------
  iotests: Disable 125 on broken XFS versions

And by that I mean all XFS versions, as far as I can tell.  All details
are in the comment below.

We never noticed this problem because we only read the first number from
qemu-img info's "disk size" output -- and that is effectively useless,
because qemu-img prints a human-readable value (which generally includes
a decimal point).  That will be fixed in the next patch.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: f2d86ade4da71d1f32ffc5977ea5417c20996919
      
https://github.com/qemu/qemu/commit/f2d86ade4da71d1f32ffc5977ea5417c20996919
  Author: Max Reitz <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/125

  Log Message:
  -----------
  iotests: Use stat -c %b in 125

125 should not use qemu-img to get the on-disk image size, because that
reports it in a human-readable format that is useless to us.  Just use
stat instead (like we do to get the image file length).

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: a6ffe1998cfe1fd3cd83de0a7d1dd16eb514f987
      
https://github.com/qemu/qemu/commit/a6ffe1998cfe1fd3cd83de0a7d1dd16eb514f987
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c
    M block/block-copy.c
    M include/block/block-copy.h

  Log Message:
  -----------
  block/backup: move in-flight requests handling from backup to block-copy

Move synchronization mechanism to block-copy, to be able to use one
block-copy instance from backup job and backup-top filter in parallel.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 843670f30f35935cfd7781b6a49c258f22261b1d
      
https://github.com/qemu/qemu/commit/843670f30f35935cfd7781b6a49c258f22261b1d
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c

  Log Message:
  -----------
  block/backup: move write_flags calculation inside backup_job_create

This is logic-less refactoring, which simplifies further patch, as
we'll need write_flags for backup-top filter creation and backup-top
should be created before block job creation.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0f4b02b73e71cb761bfedcef180c5f10812f2953
      
https://github.com/qemu/qemu/commit/0f4b02b73e71cb761bfedcef180c5f10812f2953
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup.c
    M block/block-copy.c
    M include/block/block-copy.h

  Log Message:
  -----------
  block/block-copy: split block_copy_set_callbacks function

Split block_copy_set_callbacks out of block_copy_state_new. It's needed
for further commit: block-copy will use BdrvChildren of backup-top
filter, so it will be created from backup-top filter creation function.
But callbacks will still belong to backup job and will be set in
separate.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 7df7868b9640402c465eec93c8e0fa9fc2cf5744
      
https://github.com/qemu/qemu/commit/7df7868b9640402c465eec93c8e0fa9fc2cf5744
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/Makefile.objs
    A block/backup-top.c
    A block/backup-top.h

  Log Message:
  -----------
  block: introduce backup-top filter driver

Backup-top filter caches write operations and does copy-before-write
operations.

The driver will be used in backup instead of write-notifiers.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 00e30f05de1d19586345ec373970ef4c192c6270
      
https://github.com/qemu/qemu/commit/00e30f05de1d19586345ec373970ef4c192c6270
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/backup-top.c
    M block/backup.c
    M block/block-copy.c
    M block/replication.c
    M blockdev.c
    M include/block/block-copy.h
    M include/block/block_int.h
    M qapi/block-core.json
    M tests/qemu-iotests/056
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/257
    M tests/qemu-iotests/257.out

  Log Message:
  -----------
  block/backup: use backup-top instead of write notifiers

Drop write notifiers and use filter node instead.

= Changes =

1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.

2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.

3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.

4. Block-copy is now using BdrvChildren instead of BlockBackends

5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.

= Iotest changes =

56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.

Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.

141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.

257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: e99754b42e99c33c6f3da9715c860ee542ea3806
      
https://github.com/qemu/qemu/commit/e99754b42e99c33c6f3da9715c860ee542ea3806
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M block/nbd.c

  Log Message:
  -----------
  nbd: add empty .bdrv_reopen_prepare

Fixes commit job / qemu-img commit, when
commiting qcow2 file which is based on nbd export.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1718727
Signed-off-by: Maxim Levitsky <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: da51e998f849fa7cd543e82d2301a9031fbe48e5
      
https://github.com/qemu/qemu/commit/da51e998f849fa7cd543e82d2301a9031fbe48e5
  Author: Daniel P. Berrangé <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/149
    M tests/qemu-iotests/149.out

  Log Message:
  -----------
  tests: fix I/O test for hosts defaulting to LUKSv2

Some distros are now defaulting to LUKS version 2 which QEMU cannot
process. For our I/O test that validates interoperability between the
kernel/cryptsetup and QEMU, we need to explicitly ask for version 1
of the LUKS format.

Signed-off-by: Daniel P. Berrangé <address@hidden>
Message-id: address@hidden
Tested-by: Maxim Levitsky <address@hidden>
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 35f05b2e2ee59e077bf949057dc0959ddd6e5249
      
https://github.com/qemu/qemu/commit/35f05b2e2ee59e077bf949057dc0959ddd6e5249
  Author: Max Reitz <address@hidden>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/162
    M tests/qemu-iotests/162.out

  Log Message:
  -----------
  iotests/162: Fix for newer Linux 5.3+

Linux 5.3 has made 0.0.0.0/8 a working IPv4 subnet.  As such, "42" is
now a valid host, and the connection to it will (hopefully) time out
over a long period rather than quickly return with EINVAL.

So let us use a negative integer for testing that NBD will not crash
when it receives integer hosts.  This way, the connection will again
fail quickly and reliably.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 088d67096d753a890abb25e94c57c2e594f3d236
      
https://github.com/qemu/qemu/commit/088d67096d753a890abb25e94c57c2e594f3d236
  Author: Peter Maydell <address@hidden>
  Date:   2019-10-14 (Mon, 14 Oct 2019)

  Changed paths:
    M block.c
    M block/Makefile.objs
    M block/accounting.c
    A block/aio_task.c
    A block/backup-top.c
    A block/backup-top.h
    M block/backup.c
    A block/block-copy.c
    M block/file-posix.c
    M block/nbd.c
    M block/qapi.c
    M block/qcow2.c
    M block/qcow2.h
    M block/replication.c
    M block/trace-events
    M blockdev.c
    M hw/ide/core.c
    M hw/scsi/scsi-disk.c
    M include/block/accounting.h
    A include/block/aio_task.h
    A include/block/block-copy.h
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json
    M tests/qemu-iotests/026
    M tests/qemu-iotests/026.out
    M tests/qemu-iotests/026.out.nocache
    M tests/qemu-iotests/056
    M tests/qemu-iotests/124
    M tests/qemu-iotests/125
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/149
    M tests/qemu-iotests/149.out
    M tests/qemu-iotests/162
    M tests/qemu-iotests/162.out
    M tests/qemu-iotests/227.out
    M tests/qemu-iotests/257
    M tests/qemu-iotests/257.out
    M tests/qemu-iotests/common.rc
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-10' 
into staging

Block patches:
- Parallelized request handling for qcow2
- Backup job refactoring to use a filter node instead of before-write
  notifiers
- Add discard accounting information to file-posix nodes
- Allow trivial reopening of nbd nodes
- Some iotest fixes

# gpg: Signature made Thu 10 Oct 2019 12:40:34 BST
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-10-10: (36 commits)
  iotests/162: Fix for newer Linux 5.3+
  tests: fix I/O test for hosts defaulting to LUKSv2
  nbd: add empty .bdrv_reopen_prepare
  block/backup: use backup-top instead of write notifiers
  block: introduce backup-top filter driver
  block/block-copy: split block_copy_set_callbacks function
  block/backup: move write_flags calculation inside backup_job_create
  block/backup: move in-flight requests handling from backup to block-copy
  iotests: Use stat -c %b in 125
  iotests: Disable 125 on broken XFS versions
  iotests: Fix 125 for growth_mode = metadata
  qapi: query-blockstat: add driver specific file-posix stats
  file-posix: account discard operations
  scsi: account unmap operations
  scsi: move unmap error checking to the complete callback
  scsi: store unmap offset and nb_sectors in request struct
  ide: account UNMAP (TRIM) operations
  block: add empty account cookie type
  qapi: add unmap to BlockDeviceStats
  qapi: group BlockDeviceStats fields
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/cdfc44ac3c6d...088d67096d75