[Qemu-commits] [qemu/qemu] d08980: iotests: Make 233 output more reliable

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: d08980511d78480ffbbd9897b7fd050bd03d518d
      
https://github.com/qemu/qemu/commit/d08980511d78480ffbbd9897b7fd050bd03d518d
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/233
    M tests/qemu-iotests/233.out

  Log Message:
  -----------
  iotests: Make 233 output more reliable

We have a race between the nbd server and the client both trying
to report errors at once which can make the test sometimes fail
if the output lines swap order under load.  Break the race by
collecting server messages into a file and then replaying that
at the end of the test.

We may yet want to fix the server to not output ANYTHING for a
client action except when -v was used (to avoid malicious clients
from being able to DoS a server by filling up its logs), but that
is saved for a future patch.

Signed-off-by: Eric Blake <address@hidden>
CC: Daniel P. Berrangé <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>


  Commit: ae560cc34f9ff4662d4ca1425b88fd1f85f52817
      
https://github.com/qemu/qemu/commit/ae560cc34f9ff4662d4ca1425b88fd1f85f52817
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M Makefile
    M scripts/texi2pod.pl

  Log Message:
  -----------
  maint: Allow for EXAMPLES in texi2pod

The next commit will add an EXAMPLES section to qemu-nbd.8;
for that to work, we need to recognize EXAMPLES in texi2pod.
We also need to add a dependency from all man pages against
the generator script, since a change to the generator may
cause the resulting man page to differ.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 86b7f6771f0cd1552791d1bfc2bdebd65cf967a3
      
https://github.com/qemu/qemu/commit/86b7f6771f0cd1552791d1bfc2bdebd65cf967a3
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M qemu-nbd.texi

  Log Message:
  -----------
  qemu-nbd: Enhance man page

Document some useful qemu-nbd command lines. Mention some restrictions
on particular options, like -p being only for MBR images, or -c/-d
being Linux-only.  Update some text given the recent change to no
longer serve oldstyle protocol (missed in commit 7f7dfe2a).  Also,
consistently use trailing '.' in describing options.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 4485936b6de20afa38138e9d1e8ffed88cf0be73
      
https://github.com/qemu/qemu/commit/4485936b6de20afa38138e9d1e8ffed88cf0be73
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M qemu-nbd.c

  Log Message:
  -----------
  qemu-nbd: Sanity check partition bounds

When the user requests a partition, we were using data read
from the disk as disk offsets without a bounds check. We got
lucky that even when computed offsets are out-of-bounds,
blk_pread() will gracefully catch the error later (so I don't
think a malicious image can crash or exploit qemu-nbd, and am
not treating this as a security flaw), but it's better to
flag the problem up front than to risk permanent EIO death of
the block device down the road.  The new bounds check adds
an assertion that will never fail, but rather exists to help
the compiler see that adding two positive 41-bit values
(given MBR constraints) can't overflow 64-bit off_t.

Using off_t to represent a partition length is a bit of a
misnomer; a later patch will update to saner types, but it
is left separate in case the bounds check needs to be
backported in isolation.

Also, note that the partition code blindly overwrites any
non-zero offset passed in by the user; so for now, make the
-o/-P combo an error for less confusion.  In the future, we
may let -o and -P work together (selecting a subset of a
partition); so it is okay that an explicit '-o 0' behaves
no differently from omitting -o.

This can be tested with nbdkit:
$ echo hi > file
$ nbdkit -fv --filter=truncate partitioning file truncate=64k

Pre-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
$ qemu-io -f raw nbd://localhost:10810
qemu-io> r -v 0 1
Disconnect client, due to: Failed to send reply: reading from file failed: 
Input/output error
Connection closed
read failed: Input/output error
qemu-io> q
[1]+  Done                    qemu-nbd -p 10810 -P 1 -f raw 
nbd://localhost:10809

Post-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds 
file length 65536

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>


  Commit: 7596bbb390838359e4789996f349bda0cad56b0e
      
https://github.com/qemu/qemu/commit/7596bbb390838359e4789996f349bda0cad56b0e
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M nbd/server.c

  Log Message:
  -----------
  nbd/server: Hoist length check to qmp_nbd_server_add

We only had two callers to nbd_export_new; qemu-nbd.c always
passed a valid offset/length pair (because it already checked
the file length, to ensure that offset was in bounds), while
blockdev-nbd.c always passed 0/-1.  Then nbd_export_new reduces
the size to a multiple of BDRV_SECTOR_SIZE (can only happen
when offset is not sector-aligned, since bdrv_getlength()
currently rounds up) (someday, it would be nice to have
byte-accurate lengths - but not today).

However, I'm finding it easier to work with the code if we are
consistent on having both callers pass in a valid length, and
just assert that things are sane in nbd_export_new, meaning
that no negative values were passed, and that offset+size does
not exceed 63 bits (as that really is a fundamental limit to
later operations, whether we use off_t or uint64_t).

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 9d26dfcbab62746b3e66ec7784d75c13ff499669
      
https://github.com/qemu/qemu/commit/9d26dfcbab62746b3e66ec7784d75c13ff499669
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M include/block/nbd.h
    M nbd/server.c
    M qemu-nbd.c

  Log Message:
  -----------
  nbd/server: Favor [u]int64_t over off_t

Although our compile-time environment is set up so that we always
support long files with 64-bit off_t, we have no guarantee whether
off_t is the same type as int64_t.  This requires casts when
printing values, and prevents us from directly using qemu_strtoi64()
(which will be done in the next patch). Let's just flip to uint64_t
where possible, and stick to int64_t for detecting failure of
blk_getlength(); we also keep the assertions added in the previous
patch that the resulting values fit in 63 bits.  The overflow check
in nbd_co_receive_request() was already sane (request->from is
validated to fit in 63 bits, and request->len is 32 bits, so the
addition can't overflow 64 bits), but rewrite it in a form easier
to recognize as a typical overflow check.

Rename the variable 'description' to keep line lengths reasonable.

Suggested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 43b510113bb2c6393c98a31dae9b57022a9c5636
      
https://github.com/qemu/qemu/commit/43b510113bb2c6393c98a31dae9b57022a9c5636
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M qemu-nbd.c

  Log Message:
  -----------
  qemu-nbd: Avoid strtol open-coding

Our copy-and-pasted open-coding of strtol handling forgot to
handle overflow conditions.  Use qemu_strto*() instead.

In the case of --partition, since we insist on a user-supplied
partition to be non-zero, we can use 0 rather than -1 for our
initial value to distinguish when a partition is not being
served, for slightly more optimal code.

The error messages for out-of-bounds values are less specific,
but should not be a terrible loss in quality.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>


  Commit: 091d0bf3c94737fc451a9b3f4eddf3a4d74c90b8
      
https://github.com/qemu/qemu/commit/091d0bf3c94737fc451a9b3f4eddf3a4d74c90b8
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c
    M nbd/trace-events

  Log Message:
  -----------
  nbd/client: Refactor nbd_receive_list()

Right now, nbd_receive_list() is only called by
nbd_receive_query_exports(), which in turn is only called if the
server lacks NBD_OPT_GO but has working option negotiation, and is
merely used as a quality-of-implementation trick since servers
can't give decent errors for NBD_OPT_EXPORT_NAME.  However, servers
that lack NBD_OPT_GO are becoming increasingly rare (nbdkit was a
latecomer, in Aug 2018, but qemu has been such a server since commit
f37708f6 in July 2017 and released in 2.10), so it no longer makes
sense to micro-optimize that function for performance.

Furthermore, when debugging a server's implementation, tracing the
full reply (both names and descriptions) is useful, not to mention
that upcoming patches adding 'qemu-nbd --list' will want to collect
that data.  And when you consider that a server can send an export
name up to the NBD protocol length limit of 4k; but our current
NBD_MAX_NAME_SIZE is only 256, we can't trace all valid server
names without more storage, but 4k is large enough that the heap
is better than the stack for long names.

Thus, I'm changing the division of labor, with nbd_receive_list()
now always malloc'ing a result on success (the malloc is bounded
by the fact that we reject servers with a reply length larger
than 32M), and moving the comparison to 'wantname' to the caller.

There is a minor change in behavior where a server with 0 exports
(an immediate NBD_REP_ACK reply) is now no longer distinguished
from a server without LIST support (NBD_REP_ERR_UNSUP); this
information could be preserved with a complication to the calling
contract to provide a bit more information, but I didn't see the
point.  After all, the worst that can happen if our guess at a
match is wrong is that the caller will get a cryptic disconnect
when NBD_OPT_EXPORT_NAME fails (which is no different from what
would happen if we had not tried LIST), while treating an empty
list as immediate failure would prevent connecting to really old
servers that really did lack LIST.  Besides, NBD servers with 0
exports are rare (qemu can do it when using QMP nbd-server-start
without nbd-server-add - but qemu understands NBD_OPT_GO and
thus won't tickle this change in behavior).

Fix the spelling of foundExport to match coding standards while
in the area.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 6dc1667d6881add34e9bad48ac2a848134ea8a6d
      
https://github.com/qemu/qemu/commit/6dc1667d6881add34e9bad48ac2a848134ea8a6d
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M block/nbd-client.c
    M include/block/nbd.h
    M nbd/client.c
    M nbd/trace-events
    M qemu-nbd.c

  Log Message:
  -----------
  nbd/client: Move export name into NBDExportInfo

Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over to a simplification of nbd_opt_go().

The main driver for this refactoring is that an upcoming patch
would like to add support to qemu-nbd to list information about
all exports available on a server, where the name(s) will be
provided by the server instead of the client.  But another benefit
is that we can now allow the client to explicitly specify the
empty export name "" even when connecting to an oldstyle server
(even if qemu is no longer such a server after commit 7f7dfe2a).

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 2df94eb52b68d16f8a050bc28dd94a8c7d3366ec
      
https://github.com/qemu/qemu/commit/2df94eb52b68d16f8a050bc28dd94a8c7d3366ec
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M block/nbd-client.c
    M include/block/nbd.h
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Change signature of nbd_negotiate_simple_meta_context()

Pass 'info' instead of three separate parameters related to info,
when requesting the server to set the meta context.  Update the
NBDExportInfo struct to rename the received id field to match the
fact that we are currently overloading the field to match whatever
context the user supplied through the x-dirty-bitmap hack, as well
as adding a TODO comment to remind future patches about a desire
to request two contexts at once.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 757b3ab989dea1c3143dd0d499441415ac7fcbc0
      
https://github.com/qemu/qemu/commit/757b3ab989dea1c3143dd0d499441415ac7fcbc0
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c
    M nbd/trace-events

  Log Message:
  -----------
  nbd/client: Split out nbd_send_meta_query()

Refactor nbd_negotiate_simple_meta_context() to pull out the
code that can be reused to send a LIST request for 0 or 1 query.
No semantic change.  The old comment about 'sizeof(uint32_t)'
being equivalent to '/* number of queries */' is no longer
needed, now that we are computing 'sizeof(queries)' instead.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 0182c1aed9e6a9314023c7264c5c264da2f4a4ce
      
https://github.com/qemu/qemu/commit/0182c1aed9e6a9314023c7264c5c264da2f4a4ce
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c
    M nbd/trace-events

  Log Message:
  -----------
  nbd/client: Split out nbd_receive_one_meta_context()

Extract portions of nbd_negotiate_simple_meta_context() to
a new function nbd_receive_one_meta_context() that copies the
pattern of nbd_receive_list() for performing the argument
validation of one reply.  The error message when the server
replies with more than one context changes slightly, but
that shouldn't happen in the common case.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 2b8d0954514192133b0119942edfd7a0c146900d
      
https://github.com/qemu/qemu/commit/2b8d0954514192133b0119942edfd7a0c146900d
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Refactor return of nbd_receive_negotiate()

The function could only ever return 0 or -EINVAL; make this
clearer by dropping a useless 'fail:' label.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 10b89988d6b0f5f2aed794bed5b4e774858548f4
      
https://github.com/qemu/qemu/commit/10b89988d6b0f5f2aed794bed5b4e774858548f4
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c
    M nbd/trace-events

  Log Message:
  -----------
  nbd/client: Split handshake into two functions

An upcoming patch will add the ability for qemu-nbd to list
the services provided by an NBD server.  Share the common
code of the TLS handshake by splitting the initial exchange
into a separate function, leaving only the export handling
in the original function.  Functionally, there should be no
change in behavior in this patch, although some of the code
motion may be difficult to follow due to indentation changes
(view with 'git diff -w' for a smaller changeset).

I considered an enum for the return code coordinating state
between the two functions, but in the end just settled with
ample comments.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: b3c9d33bc4e7c2ab2828b16ebc96f0414b0f1acf
      
https://github.com/qemu/qemu/commit/b3c9d33bc4e7c2ab2828b16ebc96f0414b0f1acf
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Pull out oldstyle size determination

Another refactoring creating nbd_negotiate_finish_oldstyle()
for further reuse during 'qemu-nbd --list'.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 138796d0f545ad4b6c74ad2cbe5f6e08c454a0b9
      
https://github.com/qemu/qemu/commit/138796d0f545ad4b6c74ad2cbe5f6e08c454a0b9
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c
    M nbd/trace-events

  Log Message:
  -----------
  nbd/client: Refactor nbd_opt_go() to support NBD_OPT_INFO

Rename the function to nbd_opt_info_or_go() with an added parameter
and slight changes to comments and trace messages, in order to
reuse the function for NBD_OPT_INFO.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: d21a2d3451d7f1defea5104ce28938f228fab0d4
      
https://github.com/qemu/qemu/commit/d21a2d3451d7f1defea5104ce28938f228fab0d4
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M include/block/nbd.h
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Add nbd_receive_export_list()

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing.  We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions.  Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.

This patch adds the low-level client code for grabbing the list
of exports.  It benefits from the recent refactoring patches, in
order to share as much code as possible when it comes to doing
validation of server replies.  The resulting information is stored
in an array of NBDExportInfo which has been expanded to any
description string, along with a convenience function for freeing
the list.

Note: a malicious server could exhaust memory of a client by feeding
an unending loop of exports; perhaps we should place a limit on how
many we are willing to receive. But note that a server could
reasonably be serving an export for every file in a large directory,
where an arbitrary limit in the client means we can't list anything
from such a server; the same happens if we just run until the client
fails to malloc() and thus dies by an abort(), where the limit is
no longer arbitrary but determined by available memory.  Since the
client is already planning on being short-lived, it's hard to call
this a denial of service attack that would starve off other uses,
so it does not appear to be a security issue.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 0b576b6bfb56291bb13db0a54d99adf2f3706030
      
https://github.com/qemu/qemu/commit/0b576b6bfb56291bb13db0a54d99adf2f3706030
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M include/block/nbd.h
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Add meta contexts to nbd_receive_export_list()

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing.  We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions.  Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.

This patch continues the work of the previous patch, by adding the
ability to track the list of available meta contexts into
NBDExportInfo.  It benefits from the recent refactoring patches
with a new nbd_list_meta_contexts() that reuses much of the same
framework as setting a meta context.

Note: a malicious server could exhaust memory of a client by feeding
an unending loop of contexts; perhaps we could place a limit on how
many we are willing to receive. But this is no different from our
earlier analysis on a server sending an unending list of exports,
and the death of a client due to memory exhaustion when the client
was going to exit soon anyways is not really a denial of service
attack.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 68b96f15838d309ef791cb83b5eec1bd7da271c2
      
https://github.com/qemu/qemu/commit/68b96f15838d309ef791cb83b5eec1bd7da271c2
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M qemu-nbd.c
    M qemu-nbd.texi

  Log Message:
  -----------
  qemu-nbd: Add --list option

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing.  We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions.  Thus, it is time to add
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.

This patch actually implements --list/-L, while reusing other
options such as --tls-creds for now designating how to connect
as the client (rather than their non-list usage of how to operate
as the server).

I debated about adding this functionality to something akin to
'qemu-img info' - but that tool does not readily lend itself
to connecting to an arbitrary NBD server without also tying to
a specific export (I may, however, still add ImageInfoSpecificNBD
for reporting the bitmaps available when connecting to a single
export).  And, while it may feel a bit odd that normally
qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
really making the qemu-nbd binary that much larger, because
'qemu-nbd -c' has to operate as both server and client
simultaneously across two threads when feeding the kernel module
for /dev/nbdN access.

Sample output:
$ qemu-nbd -L
exports available: 1
 export: ''
  size:  65536
  flags: 0x4ed ( flush fua trim zeroes df cache )
  min block: 512
  opt block: 4096
  max block: 33554432
  available meta contexts: 1
   base:allocation

Note that the output only lists sizes if the server sent
NBD_FLAG_HAS_FLAGS, because a newstyle server does not give
the size otherwise.  It has the side effect that for really
old servers that did not send any flags, the size is not
output even though it was available.  However, I'm not too
concerned about that - oldstyle servers are (rightfully)
getting less common to encounter (qemu 3.0 was the last
version where we even serve it), and most existing servers
that still even offer oldstyle negotiation (such as nbdkit)
still send flags (since that was added to the NBD protocol
in 2007 to permit read-only connections).

Not done here, but maybe worth future experiments: capture
the meat of NBDExportInfo into a QAPI struct, and use the
generated QAPI pretty-printers instead of hand-rolling our
output loop.  It would also permit us to add a JSON output
mode for machine parsing.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 7c6f5ddca62905e67025aa0657e8a011cbdffa11
      
https://github.com/qemu/qemu/commit/7c6f5ddca62905e67025aa0657e8a011cbdffa11
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M nbd/client.c

  Log Message:
  -----------
  nbd/client: Work around 3.0 bug for listing meta contexts

Commit 3d068aff forgot to advertise available qemu: contexts
when the client requests a list with 0 queries. Furthermore,
3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
216ee365) that _silently_ acts as though the entire image is
clean if a requested bitmap is not present.  Both bugs have
been recently fixed, so that a modern qemu server gives full
context output right away, and the client refuses a
connection if a requested x-dirty-bitmap was not found.

Still, it is likely that there will be users that have to
work with a mix of old and new qemu versions, depending on
which features get backported where, at which point being
able to rely on 'qemu-img --list' output to know for sure
whether a given NBD export has the desired dirty bitmap is
much nicer than blindly connecting and risking that the
entire image may appear clean.  We can make our --list code
smart enough to work around buggy servers by tracking
whether we've seen any qemu: replies in the original 0-query
list; if not, repeat with a single query on "qemu:" (which
may still have no replies, but then we know for sure we
didn't trip up on the server bug).

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: ddd09448fd833d646952c769ae9ce3d39bee989f
      
https://github.com/qemu/qemu/commit/ddd09448fd833d646952c769ae9ce3d39bee989f
  Author: Eric Blake <address@hidden>
  Date:   2019-01-21 (Mon, 21 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/233
    M tests/qemu-iotests/233.out

  Log Message:
  -----------
  iotests: Enhance 223, 233 to cover 'qemu-nbd --list'

Any good new feature deserves some regression testing :)
Coverage includes:
- 223: what happens when there are 0 or more than 1 export,
proof that we can see multiple contexts including qemu:dirty-bitmap
- 233: proof that we can list over TLS, and that mix-and-match of
plain/TLS listings will behave sanely

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Tested-by: Richard W.M. Jones <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 952bc8b3c2cbba78261923a1e8ca55cda261dee9
      
https://github.com/qemu/qemu/commit/952bc8b3c2cbba78261923a1e8ca55cda261dee9
  Author: Peter Maydell <address@hidden>
  Date:   2019-01-22 (Tue, 22 Jan 2019)

  Changed paths:
    M Makefile
    M block/nbd-client.c
    M blockdev-nbd.c
    M include/block/nbd.h
    M nbd/client.c
    M nbd/server.c
    M nbd/trace-events
    M qemu-nbd.c
    M qemu-nbd.texi
    M scripts/texi2pod.pl
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/233
    M tests/qemu-iotests/233.out

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-21' into 
staging

nbd patches for 2019-01-21

Add 'qemu-nbd --list' for probing a remote NBD server's advertisements.

- Eric Blake: 0/21 nbd: add qemu-nbd --list

# gpg: Signature made Mon 21 Jan 2019 22:44:27 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <address@hidden>"
# gpg:                 aka "Eric Blake (Free Software Programmer) 
<address@hidden>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-21: (21 commits)
  iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
  nbd/client: Work around 3.0 bug for listing meta contexts
  qemu-nbd: Add --list option
  nbd/client: Add meta contexts to nbd_receive_export_list()
  nbd/client: Add nbd_receive_export_list()
  nbd/client: Refactor nbd_opt_go() to support NBD_OPT_INFO
  nbd/client: Pull out oldstyle size determination
  nbd/client: Split handshake into two functions
  nbd/client: Refactor return of nbd_receive_negotiate()
  nbd/client: Split out nbd_receive_one_meta_context()
  nbd/client: Split out nbd_send_meta_query()
  nbd/client: Change signature of nbd_negotiate_simple_meta_context()
  nbd/client: Move export name into NBDExportInfo
  nbd/client: Refactor nbd_receive_list()
  qemu-nbd: Avoid strtol open-coding
  nbd/server: Favor [u]int64_t over off_t
  nbd/server: Hoist length check to qmp_nbd_server_add
  qemu-nbd: Sanity check partition bounds
  qemu-nbd: Enhance man page
  maint: Allow for EXAMPLES in texi2pod
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/851aa0a5a89b...952bc8b3c2cb
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.