Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#2] Dissecting QEMU Into Three Main Parts

[Lukáš Doktor]

Dne 29. 06. 20 v 12:25 Ahmed Karaman napsal(a):
> Hi,
> 
> The second report of the TCG Continuous Benchmarking series builds
> upon the QEMU performance metrics calculated in the previous report.
> This report presents a method to dissect the number of instructions
> executed by a QEMU invocation into three main phases:
> - Code Generation
> - JIT Execution
> - Helpers Execution
> It devises a Python script that automates this process.
> 
> After that, the report presents an experiment for comparing the
> output of running the script on 17 different targets. Many conclusions
> can be drawn from the results and two of them are discussed in the
> analysis section.
> 
> Report link:
> https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Dissecting-QEMU-Into-Three-Main-Parts/
> 
> Previous reports:
> Report 1 - Measuring Basic Performance Metrics of QEMU:
> https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06692.html
> 
> Best regards,
> Ahmed Karaman

Hello Ahmed,

very nice reading, both reports so far. One thing that could be better 
displayed is the system you used this to generate. This would come handy 
especially later when you move from examples to actual reports. I think it'd 
make sense to add a section with a clear definition of the machine as well as 
the operation system, qemu version and eventually other deps (like compiler, 
flags, ...). For this report something like:

architecture: x86_64
cpu_codename: Kaby Lake
cpu: i7-8650U
ram: 32GB DDR4
os: Fedora 32
qemu: 470dd165d152ff7ceac61c7b71c2b89220b3aad7
compiler: gcc-10.1.1-1.fc32.x86_64
flags: 
--target-list="x86_64-softmmu,ppc64-softmmu,aarch64-softmmu,s390x-softmmu,riscv64-softmmu"
 --disable-werror --disable-sparse --enable-sdl --enable-kvm  
--enable-vhost-net --enable-vhost-net --enable-attr  --enable-kvm  --enable-fdt 
  --enable-vnc --enable-seccomp 
--block-drv-rw-whitelist="vmdk,null-aio,quorum,null-co,blkverify,file,nbd,raw,blkdebug,host_device,qed,nbd,iscsi,gluster,rbd,qcow2,throttle,copy-on-read"
 --python=/usr/bin/python3 --enable-linux-io-uring

would do. Maybe it'd be even a good idea to create a script to report this 
basic set of information and add it after each of the perf scripts so people 
don't forget to double-check the conditions, but others might disagree so take 
this only as a suggestion.

Regards,
Lukáš

PS: Automated cpu codenames, hosts OSes and such could be tricky, but one can 
use other libraries or just best-effort-approach with fallback to "unknown" to 
let people filling it manually or adding their branch to your script.

Regards,
Lukáš

signature.asc
Description: OpenPGP digital signature