-warmup

[Ahmet Göksu]

Hi,
I have updated the bench code. 
It is warming up as the given number of seconds with -w flag before every benchmark test.

There are still differences like 100%..
Also, 1 sec warmup means (test count)*(font count) 70 secs for the results.  

I am thinking of what else can be done and waiting for your test.

Best,

Goksu

goksu.in

On 3 Aug 2023 05:45 +0300, Werner LEMBERG <wl@gnu.org>, wrote:

I have done the changes you want.

Thanks!

36.5% run difference is baaaad. AFAICS, you haven't yet worked on
omitting 'warmup' iterations, right?

I am planning to increase the iteration count by 10% and ignore the
results for them.

You mean you are going to ignore the first 10% of the iterations?
This might be a good default. However, I still think that a
`--warmup` command-line option makes sense to control this.

Trying to figure out the benchmarking program but actually drowning
in 1500 lines of code.

:-) I hope you can eventually find what you need.


Werner

Freetype Benchmark Results

Warning: Baseline and Benchmark have the same commit ID!

Info

Info	Baseline	Benchmark
Parameters	-c 500 -w 1	-c 500 -w 1
Commit ID	fae5e8c9	fae5e8c9
Commit Date	2023-08-01 17:37:55 +0300	2023-08-01 17:37:55 +0300
Branch	GSoC-2023-Ahmet	GSoC-2023-Ahmet

* Smaller values mean faster operation

Results for Roboto_subset.ttf

Test	N	* Baseline (ms)	* Benchmark (ms)	Difference (%)
Load	59880	0.250	0.270	-8.0
Load_Advances (Normal)	59880	0.311	0.243	21.9
Load_Advances (Fast)	59880	0.002	0.002	0.0
Load_Advances (Unscaled)	59880	0.001	0.002	-100.0
Render	59880	0.220	0.218	0.9
Get_Glyph	59880	0.086	0.086	0.0
Get_Char_Index	46906	0.001	0.001	0.0
Iterate CMap	499	0.001	0.001	0.0
New_Face	499	0.022	0.022	0.0
Embolden	59880	0.115	0.115	0.0
Stroke	57120	1.628	1.629	-0.1
Get_BBox	59880	0.055	0.055	0.0
Get_CBox	59880	0.043	0.042	2.3
New_Face & load glyph(s)	59880	0.053	0.054	-1.9
TOTAL	1407648	2.788	2.740	-6.1

Results for Arial_subset.ttf

Test	N	* Baseline (ms)	* Benchmark (ms)	Difference (%)
Load	47405	0.390	0.376	3.6
Load_Advances (Normal)	47405	0.344	0.340	1.2
Load_Advances (Fast)	47405	0.001	0.001	0.0
Load_Advances (Unscaled)	47405	0.001	0.001	0.0
Render	47405	0.179	0.186	-3.9
Get_Glyph	47405	0.075	0.076	-1.3
Get_Char_Index	46906	0.001	0.001	0.0
Iterate CMap	499	0.001	0.001	0.0
New_Face	499	0.028	0.025	10.7
Embolden	47405	0.103	0.103	0.0
Stroke	47405	1.324	1.329	-0.4
Get_BBox	47405	0.049	0.051	-4.1
Get_CBox	47405	0.035	0.036	-2.9
New_Face & load glyph(s)	47405	0.060	0.054	10.0
TOTAL	1138718	2.591	2.580	0.9

Results for TimesNewRoman_subset.ttf

Test	N	* Baseline (ms)	* Benchmark (ms)	Difference (%)
Load	47405	0.455	0.464	-2.0
Load_Advances (Normal)	47405	0.419	0.412	1.7
Load_Advances (Fast)	47405	0.001	0.001	0.0
Load_Advances (Unscaled)	47405	0.001	0.001	0.0
Render	47405	0.201	0.197	2.0
Get_Glyph	47405	0.078	0.082	-5.1
Get_Char_Index	46906	0.001	0.001	0.0
Iterate CMap	499	0.001	0.001	0.0
New_Face	499	0.027	0.027	0.0
Embolden	47405	0.140	0.139	0.7
Stroke	40185	1.545	1.545	0.0
Get_BBox	47405	0.058	0.059	-1.7
Get_CBox	47405	0.037	0.041	-10.8
New_Face & load glyph(s)	47405	0.069	0.076	-10.1
TOTAL	1124278	3.033	3.046	-1.8

Results for Tahoma_subset.ttf

Test	N	* Baseline (ms)	* Benchmark (ms)	Difference (%)
Load	47405	0.265	0.289	-9.1
Load_Advances (Normal)	47405	0.245	0.239	2.4
Load_Advances (Fast)	47405	0.001	0.001	0.0
Load_Advances (Unscaled)	47405	0.001	0.001	0.0
Render	47405	0.166	0.166	0.0
Get_Glyph	47405	0.075	0.071	5.3
Get_Char_Index	46906	0.001	0.001	0.0
Iterate CMap	499	0.001	0.001	0.0
New_Face	499	0.025	0.024	4.0
Embolden	47405	0.118	0.108	8.5
Stroke	47405	1.244	1.236	0.6
Get_BBox	47405	0.047	0.047	0.0
Get_CBox	47405	0.034	0.034	0.0
New_Face & load glyph(s)	47405	0.053	0.052	1.9
TOTAL	1138718	2.276	2.270	1.0

Results for Verdana_subset.ttf

Test	N	* Baseline (ms)	* Benchmark (ms)	Difference (%)
Load	47405	0.251	0.220	12.4
Load_Advances (Normal)	47405	0.226	0.217	4.0
Load_Advances (Fast)	47405	0.001	0.001	0.0
Load_Advances (Unscaled)	47405	0.002	0.001	50.0
Render	47405	0.169	0.174	-3.0
Get_Glyph	47405	0.073	0.075	-2.7
Get_Char_Index	46906	0.001	0.001	0.0
Iterate CMap	499	0.001	0.001	0.0
New_Face	499	0.024	0.023	4.2
Embolden	47405	0.103	0.104	-1.0
Stroke	47405	1.230	1.319	-7.2
Get_BBox	47405	0.046	0.050	-8.7
Get_CBox	47405	0.034	0.040	-17.6
New_Face & load glyph(s)	47405	0.050	0.056	-12.0
TOTAL	1138718	2.211	2.282	1.3

Freetype Benchmark