Re: [REPORT] Nightly Performance Tests - Sunday, September 13, 2020

[Ahmed Karaman]

On Mon, Sep 14, 2020 at 8:46 AM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>
> Hi Ahmed,
>
> On 9/14/20 12:07 AM, Ahmed Karaman wrote:
> > Host CPU         : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
> > Host Memory      : 15.49 GB
> >
> > Start Time (UTC) : 2020-09-13 21:35:01
> > End Time (UTC)   : 2020-09-13 22:07:44
> > Execution Time   : 0:32:42.230467
> >
> > Status           : SUCCESS
> >
> > Note:
> > Changes denoted by '-----' are less than 0.01%.
> >
> > --------------------------------------------------------
> >             SUMMARY REPORT - COMMIT f00f57f3
> > --------------------------------------------------------
>
> (Maybe this was already commented earlier but I missed it).
>
> What change had a so significant impact on the m68k target?
> At a glance I only see mostly changes in softfloat:
>
> $ git log --oneline v5.1.0..f00f57f3 tcg target/m68k fpu
> fe4b0b5bfa9 tcg: Implement 256-bit dup for tcg_gen_gvec_dup_mem
> 6a17646176e tcg: Eliminate one store for in-place 128-bit dup_mem
> e7e8f33fb60 tcg: Fix tcg gen for vectorized absolute value
> 5ebf5f4be66 softfloat: Define misc operations for bfloat16
> 34f0c0a98a5 softfloat: Define convert operations for bfloat16
> 8282310d853 softfloat: Define operations for bfloat16
> 0d93d8ec632 softfloat: Add fp16 and uint8/int8 conversion functions
> fbcc38e4cb1 softfloat: add xtensa specialization for pickNaNMulAdd
> 913602e3ffe softfloat: pass float_status pointer to pickNaN
> cc43c692511 softfloat: make NO_SIGNALING_NANS runtime property
> 73ebe95e8e5 target/ppc: add vmulld to INDEX_op_mul_vec case
>
> > --------------------------------------------------------
> > --------------------------------------------------------
> > Test Program: matmult_double
> > --------------------------------------------------------
> > Target              Instructions      Latest      v5.1.0
> > ----------  --------------------  ----------  ----------
> > aarch64            1 412 412 599       -----     +0.311%
> > alpha              3 233 957 639       -----     +7.472%
> > arm                8 545 302 995       -----      +1.09%
> > hppa               3 483 527 330       -----     +4.466%
> > m68k               3 919 110 506       -----    +18.433%
> > mips               2 344 641 840       -----     +4.085%
> > mipsel             3 329 912 425       -----     +5.177%
> > mips64             2 359 024 910       -----     +4.075%
> > mips64el           3 343 650 686       -----     +5.166%
> > ppc                3 209 505 701       -----     +3.248%
> > ppc64              3 287 495 266       -----     +3.173%
> > ppc64le            3 287 135 580       -----     +3.171%
> > riscv64            1 221 617 903       -----     +0.278%
> > s390x              2 874 160 417       -----     +5.826%
> > sh4                3 544 094 841       -----      +6.42%
> > sparc64            3 426 094 848       -----     +7.138%
> > x86_64             1 249 076 697       -----     +0.335%
> > --------------------------------------------------------
> ...
> > --------------------------------------------------------
> > Test Program: qsort_double
> > --------------------------------------------------------
> > Target              Instructions      Latest      v5.1.0
> > ----------  --------------------  ----------  ----------
> > aarch64            2 709 839 947       -----     +2.423%
> > alpha              1 969 432 086       -----     +3.679%
> > arm                8 323 168 267       -----     +2.589%
> > hppa               3 188 316 726       -----       +2.9%
> > m68k               4 953 947 225       -----    +15.153%
> > mips               2 123 789 120       -----     +3.049%
> > mipsel             2 124 235 492       -----     +3.049%
> > mips64             1 999 025 951       -----     +3.404%
> > mips64el           1 996 433 190       -----     +3.409%
> > ppc                2 819 299 843       -----     +5.436%
> > ppc64              2 768 177 037       -----     +5.512%
> > ppc64le            2 724 766 044       -----     +5.602%
> > riscv64            1 638 324 190       -----     +4.021%
> > s390x              2 519 117 806       -----     +3.364%
> > sh4                2 595 696 102       -----       +3.0%
> > sparc64            3 988 892 763       -----     +2.744%
> > x86_64             2 033 624 062       -----     +3.242%
> > --------------------------------------------------------

Hi Mr. Philippe,
The performance degradation from v5.1.0 of all targets, and especially
m68k, was introduced between the two nightly tests below:

[REPORT] Nightly Performance Tests - Thursday, August 20, 2020:
https://lists.nongnu.org/archive/html/qemu-devel/2020-08/msg04923.html

[REPORT] Nightly Performance Tests - Saturday, August 22, 2020
https://lists.nongnu.org/archive/html/qemu-devel/2020-08/msg05537.html

It looks like the new build system is the culprit.

The "bisect.py" script introduced during the "TCG Continuous
Benchmarking" GSoC project can be very handy in these cases. I wrote
about the tool and how to use it in the report below:
https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Finding-Commits-Affecting-QEMU-Performance/

Best regards,
Ahmed Karaman