[Qemu-devel] I/O accounting overhaul

[Alberto Garcia]

Hello,

I would like to retake the work that Benoît was about to start last
year and extend the I/O accounting in QEMU. I was reading the past
discussions and I will try to summarize all the ideas.

The current accounting code collects the following information:

  typedef struct BlockAcctStats {
      uint64_t nr_bytes[BLOCK_MAX_IOTYPE];
      uint64_t nr_ops[BLOCK_MAX_IOTYPE];
      uint64_t total_time_ns[BLOCK_MAX_IOTYPE];
      uint64_t merged[BLOCK_MAX_IOTYPE];
      uint64_t wr_highest_sector;
  } BlockAcctStats;

where the arrays hold information for read, write and flush
operations.

The accounting stats are stored in the BlockDriverState, but they're
actually from the device backed by the BDS, so they could probably be
moved there. For the interface we could extend BlockDeviceStats and
add the new fields, but query-blockstats works on BDS, so maybe we
need new API?

The fields are mostly self-explanatory. merged counts the number of
requests merged into a single one (using virtio_blk_submit_multireq),
and wr_highest_sector is the number of the highest sector that has
been written.

In addition to those we can have:

uint64_t nr_invalid_ops[BLOCK_MAX_IOTYPE];
uint64_t nr_failed_ops[BLOCK_MAX_IOTYPE];

   The decision about whether to count these two as done (for e.g.
   total_time_ns) could be configurable by the user.

int64_t last_access_time_ns;

   This would be updated after each operation, and would be useful to
   know for how long a particular device has been idle.

uint64_t latency[BLOCK_MAX_IOTYPE];

   What we added in average to total_time_ns[] in the past second (or
   minute, or hour; the interval would be configurable). We could also
   collect the maximum and minimum latencies for that period.

   This could be updated every time an operation is accounted, so I
   think it could be implemented without adding any timer.

uint64_t queue_depth[BLOCK_MAX_IOTYPE];

   Average number of requests. Similar to the previous one. It would
   require us to keep a count of ongoing requests as well.

About the implementation, I read that it was possible to call
block_acct_start() without calling block_acct_done(). I don't know if
that's still the case, I need to check that.

I don't know if I'm forgetting anything. I have a rough implementation
covering most of the things I described, but of course it needs to be
polished etc. before publishing.

What do you think about this? Comments and suggestions are welcome.

Thanks,

Berto