|
| From: | Aleksandar Markovic |
| Subject: | Re: [PATCH 7/9] tests/performance: Add nightly tests |
| Date: | Wed, 2 Sep 2020 10:36:47 +0200 |
A nightly performance testing system to monitor any change in QEMU
performance across seventeen different targets.
The system includes eight different benchmarks to provide a variety
of testing workloads.
dijkstra_double:
Find the shortest path between the source node and all other nodes
using Dijkstra’s algorithm. The graph contains n nodes where all nxn
distances are double values. The value of n can be specified using
the -n flag. The default value is 2000.
dijkstra_int32:
Find the shortest path between the source node and all other nodes
using Dijkstra’s algorithm. The graph contains n nodes where all nxn
distances are int32 values. The value of n can be specified using
the -n flag. The default value is 2000.
matmult_double:
Standard matrix multiplication of an n*n matrix of randomly generated
double numbers from 0 to 100. The value of n is passed as an argument
with the -n flag. The default value is 200.
matmult_int32:
Standard matrix multiplication of an n*n matrix of randomly generated
integer numbers from 0 to 100. The value of n is passed as an
argument with the -n flag. The default value is 200.
qsort_double:
Quick sort of an array of n randomly generated double numbers from 0
to 1000. The value of n is passed as an argument with the -n flag.
The default value is 300000.
qsort_int32:
Quick sort of an array of n randomly generated integer numbers from 0
to 50000000. The value of n is passed as an argument with the -n
flag.The default value is 300000.
qsort_string:
Quick sort of an array of 10000 randomly generated strings of size 8
(including null terminating character). The sort process is repeated
n number of times. The value of n is passed as an argument with the
-n flag. The default value is 20.
search_string:
Search for the occurrence of a small string in a much larger random
string (“needle in a hay”). The search process is repeated n number
of times and each time, a different large random string (“hay”) is
generated. The value of n can be specified using the -n flag. The
default value is 20.
Syntax:
nightly_tests_core.py [-h] [-r REF]
Optional arguments:
-h, --help Show this help message and exit
-r REF, --reference REF
Reference QEMU version - Default is v5.1.0
Example of usage:
nightly_tests_core.py -r v5.1.0 2>log.txt
The following report includes detailed setup and execution details
of the system:
https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/ QEMU-Nightly-Performance- Tests/
Signed-off-by: Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
---
tests/performance/nightly-tests/README.md | 243 +++++
.../source/dijkstra_double/dijkstra_double.c | 194 ++++
.../source/dijkstra_int32/dijkstra_int32.c | 192 ++++
.../source/matmult_double/matmult_double.c | 123 +++
.../source/matmult_int32/matmult_int32.c | 121 +++
.../source/qsort_double/qsort_double.c | 104 ++
.../source/qsort_int32/qsort_int32.c | 103 ++
.../source/qsort_string/qsort_string.c | 122 +++
.../source/search_string/search_string.c | 110 +++
.../scripts/nightly_tests_core.py | 920 ++++++++++++++++++
.../scripts/run_nightly_tests.py | 135 +++
.../nightly-tests/scripts/send_email.py | 56 ++
12 files changed, 2423 insertions(+)
create mode 100644 tests/performance/nightly-tests/README.md
create mode 100644 tests/performance/nightly-tests/benchmarks/source/ dijkstra_double/dijkstra_ double.c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/ dijkstra_int32/dijkstra_int32. c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/ matmult_double/matmult_double. c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/ matmult_int32/matmult_int32.c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_ double/qsort_double.c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_ int32/qsort_int32.c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_ string/qsort_string.c
create mode 100644 tests/performance/nightly-tests/benchmarks/source/ search_string/search_string.c
create mode 100755 tests/performance/nightly-tests/scripts/nightly_tests_ core.py
create mode 100755 tests/performance/nightly-tests/scripts/run_nightly_ tests.py
create mode 100644 tests/performance/nightly-tests/scripts/send_email.py
diff --git a/tests/performance/nightly-tests/README.md b/tests/performance/nightly- tests/README.md
new file mode 100644
index 0000000000..6db3b351b3
--- /dev/null
+++ b/tests/performance/nightly-tests/README.md
@@ -0,0 +1,243 @@
+### QEMU Nightly Tests
+
+**Required settings:**
+
+Update the `GMAIL_USER` object in `send_email.py` with your credentials.
+
+For more details on how the system works, please check the [eighth report](https://ahmedkrmn.github.io/TCG-Continuous- ) of the "TCG Continuos Benchmarking" series.Benchmarking/QEMU-Nightly- Performance-Tests/
+
+**Running the System:**
+
+The default reference version is v5.1.0. To specify a custom version, please use the `-r, --reference` flag.
+
+```bash
+./run_nightly_tests.py
+```
+
+**Output:**
+
+```
+Host CPU : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+Host Memory : 15.49 GB
+
+Start Time (UTC) : 2020-08-25 21:30:01
+End Time (UTC) : 2020-08-25 22:02:37
+Execution Time : 0:32:35.896990
+
+Status : SUCCESS
+
+Note:
+Changes denoted by '-----' are less than 0.01%.
+
+--------------------------------------------------------
+ SUMMARY REPORT - COMMIT d1a2b51f
+--------------------------------------------------------
+ AVERAGE RESULTS
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 158 355 274 ----- +1.693%
+alpha 1 914 967 171 ----- +3.524%
+arm 8 076 402 940 ----- +2.304%
+hppa 4 261 685 987 -0.182% +3.164%
+m68k 2 690 273 044 ----- +7.131%
+mips 1 862 033 667 ----- +2.494%
+mipsel 2 008 211 069 ----- +2.674%
+mips64 1 918 635 565 ----- +2.818%
+mips64el 2 051 565 677 ----- +3.026%
+ppc 2 480 141 217 ----- +3.107%
+ppc64 2 576 713 959 ----- +3.143%
+ppc64le 2 558 853 539 ----- +3.173%
+riscv64 1 406 704 050 ----- +2.65%
+s390x 3 158 140 046 ----- +3.118%
+sh4 2 364 449 748 ----- +3.33%
+sparc64 3 318 544 783 ----- +3.851%
+x86_64 1 775 844 158 ----- +2.156%
+--------------------------------------------------------
+
+ DETAILED RESULTS
+--------------------------------------------------------
+Test Program: dijkstra_double
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 3 062 583 464 ----- +1.424%
+alpha 3 191 864 698 ----- +3.696%
+arm 16 357 157 526 ----- +2.347%
+hppa 7 228 376 315 -0.139% +3.086%
+m68k 4 294 016 587 ----- +9.692%
+mips 3 051 419 166 ----- +2.427%
+mipsel 3 231 509 618 ----- +2.869%
+mips64 3 245 837 754 ----- +2.596%
+mips64el 3 414 195 796 ----- +3.021%
+ppc 4 914 520 972 -0.041% +4.74%
+ppc64 5 098 154 311 ----- +4.565%
+ppc64le 5 082 419 054 ----- +4.58%
+riscv64 2 192 294 915 ----- +1.955%
+s390x 4 584 503 977 ----- +2.896%
+sh4 3 949 036 447 ----- +3.464%
+sparc64 4 586 203 546 ----- +4.237%
+x86_64 2 484 092 105 ----- +1.75%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: dijkstra_int32
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 210 194 577 ----- +1.493%
+alpha 1 494 133 274 ----- +2.15%
+arm 8 262 935 967 ----- +2.665%
+hppa 5 207 318 306 ----- +3.047%
+m68k 1 725 856 962 ----- +2.527%
+mips 1 495 227 032 ----- +1.492%
+mipsel 1 497 147 869 ----- +1.479%
+mips64 1 715 388 570 ----- +1.892%
+mips64el 1 695 276 864 ----- +1.913%
+ppc 2 014 557 389 ----- +1.819%
+ppc64 2 206 267 901 ----- +2.139%
+ppc64le 2 197 998 781 ----- +2.146%
+riscv64 1 354 912 745 ----- +2.396%
+s390x 2 916 247 062 ----- +1.241%
+sh4 1 990 532 533 ----- +2.669%
+sparc64 2 872 231 051 ----- +3.758%
+x86_64 1 553 981 241 ----- +2.12%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: matmult_double
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 1 412 273 223 ----- +0.302%
+alpha 3 233 991 649 ----- +7.473%
+arm 8 545 173 979 ----- +1.088%
+hppa 3 483 597 802 -1.267% +4.468%
+m68k 3 919 065 529 ----- +18.431%
+mips 2 344 774 894 ----- +4.091%
+mipsel 3 329 886 464 ----- +5.177%
+mips64 2 359 046 988 ----- +4.076%
+mips64el 3 343 664 785 ----- +5.167%
+ppc 3 209 457 051 ----- +3.246%
+ppc64 3 287 503 981 ----- +3.173%
+ppc64le 3 287 189 065 ----- +3.173%
+riscv64 1 221 603 682 ----- +0.277%
+s390x 2 874 199 923 ----- +5.827%
+sh4 3 543 943 634 ----- +6.416%
+sparc64 3 426 150 004 ----- +7.139%
+x86_64 1 248 917 276 ----- +0.322%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: matmult_int32
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 598 681 621 ----- +0.585%
+alpha 372 437 418 ----- +0.677%
+arm 746 583 193 ----- +1.462%
+hppa 674 278 359 ----- +1.183%
+m68k 410 495 553 ----- +0.9%
+mips 499 698 837 ----- +0.531%
+mipsel 499 500 429 ----- +0.497%
+mips64 481 554 664 ----- +0.599%
+mips64el 465 057 054 ----- +0.619%
+ppc 341 334 603 ----- +0.944%
+ppc64 393 796 203 ----- +0.966%
+ppc64le 393 977 298 ----- +0.965%
+riscv64 351 709 769 ----- +0.785%
+s390x 494 427 384 ----- +0.599%
+sh4 402 668 444 ----- +0.899%
+sparc64 495 952 959 ----- +1.192%
+x86_64 402 928 461 ----- +0.833%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: qsort_double
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 709 683 624 ----- +2.417%
+alpha 1 969 460 172 ----- +3.68%
+arm 8 322 998 390 ----- +2.587%
+hppa 3 188 301 995 -0.047% +2.9%
+m68k 4 953 930 065 ----- +15.153%
+mips 2 123 919 587 ----- +3.055%
+mipsel 2 124 212 187 ----- +3.048%
+mips64 1 999 047 826 ----- +3.405%
+mips64el 1 996 426 772 ----- +3.409%
+ppc 2 819 267 902 -0.021% +5.435%
+ppc64 2 768 186 548 ----- +5.513%
+ppc64le 2 724 803 772 -0.011% +5.603%
+riscv64 1 638 328 937 ----- +4.021%
+s390x 2 519 081 708 ----- +3.362%
+sh4 2 595 545 154 ----- +2.994%
+sparc64 3 988 986 587 ----- +2.747%
+x86_64 2 033 468 588 ----- +3.234%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: qsort_int32
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 193 392 565 ----- +2.916%
+alpha 1 521 291 933 ----- +4.193%
+arm 3 465 445 043 ----- +2.756%
+hppa 2 280 034 340 ----- +3.821%
+m68k 1 843 189 041 ----- +3.583%
+mips 1 558 024 873 ----- +3.863%
+mipsel 1 560 583 980 ----- +3.846%
+mips64 1 563 415 749 ----- +4.412%
+mips64el 1 542 677 320 ----- +4.474%
+ppc 1 728 698 880 ----- +3.665%
+ppc64 1 842 444 545 ----- +3.555%
+ppc64le 1 791 822 067 ----- +3.661%
+riscv64 1 348 866 430 ----- +4.654%
+s390x 2 184 073 151 ----- +3.319%
+sh4 1 946 492 539 ----- +3.624%
+sparc64 3 452 215 585 ----- +2.937%
+x86_64 1 813 544 414 ----- +3.537%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: qsort_string
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 592 218 418 ----- +2.468%
+alpha 1 855 834 626 ----- +3.487%
+arm 7 347 721 165 ----- +2.682%
+hppa 4 758 753 926 ----- +3.543%
+m68k 2 376 811 462 ----- +3.567%
+mips 2 166 608 045 ----- +2.532%
+mipsel 2 163 392 541 ----- +2.528%
+mips64 2 029 251 969 ----- +3.117%
+mips64el 2 011 628 621 ----- +3.145%
+ppc 2 492 942 463 ----- +2.673%
+ppc64 2 464 702 554 ----- +2.488%
+ppc64le 2 445 253 307 ----- +2.505%
+riscv64 1 625 053 328 ----- +3.953%
+s390x 4 194 608 798 ----- +6.623%
+sh4 2 164 142 539 ----- +3.166%
+sparc64 4 299 516 539 ----- +4.065%
+x86_64 2 940 456 780 ----- +2.649%
+--------------------------------------------------------
+--------------------------------------------------------
+Test Program: search_string
+--------------------------------------------------------
+Target Instructions Latest v5.1.0
+---------- -------------------- ---------- ----------
+aarch64 2 487 814 704 ----- +1.94%
+alpha 1 680 723 605 ----- +2.835%
+arm 11 563 208 260 ----- +2.848%
+hppa 7 272 826 858 ----- +3.263%
+m68k 1 998 819 159 ----- +3.198%
+mips 1 656 596 909 ----- +1.959%
+mipsel 1 659 455 464 ----- +1.947%
+mips64 1 955 541 001 ----- +2.447%
+mips64el 1 943 598 207 ----- +2.462%
+ppc 2 320 350 477 ----- +2.332%
+ppc64 2 552 655 634 ----- +2.742%
+ppc64le 2 547 364 971 ----- +2.748%
+riscv64 1 520 862 601 ----- +3.159%
+s390x 5 497 978 370 ----- +1.078%
+sh4 2 323 236 696 ----- +3.41%
+sparc64 3 427 101 999 ----- +4.73%
+x86_64 1 729 364 402 ----- +2.806%
+--------------------------------------------------------
+```
diff --git a/tests/performance/nightly-tests/benchmarks/source/ dijkstra_double/dijkstra_ double.c b/tests/performance/nightly- tests/benchmarks/source/ dijkstra_double/dijkstra_ double.c
new file mode 100644
index 0000000000..9c0bb804ac
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/ dijkstra_double/dijkstra_ double.c
@@ -0,0 +1,194 @@
+/*
+ * Source file of a benchmark program involving calculations of the
+ * shortest distances between a source node and all other nodes in a
+ * graph of n nodes in which all nxn distances are defined as "double".
+ * The number n can be given via command line, and the default is 2000.
+ * The algorithm used is Dijsktra's.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <float.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of columns and rows in all matrixes*/
+#define DEFAULT_NODE_COUNT 2000
+#define MIN_NODE_COUNT 3
+#define MAX_NODE_COUNT 10000
+
+
+int32_t closest_index(int32_t count, double *distances, bool *flags)
+{
+ int32_t closest;
+ double minimum = DBL_MAX;
+
+ for (size_t i = 0; i < count; i++) {
+ if (flags[i] == false && distances[i] <= minimum) {
+ closest = i;
+ minimum = distances[i];
+ }
+ }
+
+ return closest;
+}
+
+/**
+ * Calculate the shortest distances from the source node using Dijkstra method.
+ * @param (out) distances An array of shortest distances from the source node.
+ * @param (out) via An array of nodes needed to be taken as the the last
+ * before destination, for each destination.
+ * @param (out) eccent Eccentricity of the source node.
+ * @param (in) count The number of nodes.
+ * @param (in) source Source node.
+ * @param (in) matrix Distance matrix.
+ */
+void find_shortest_distances(double *distances, int32_t *via, double *eccent,
+ int32_t count, int32_t source, double **matrix)
+{
+ bool *flags;
+
+ flags = (bool *)malloc(count * sizeof(bool));
+
+ for (size_t i = 0; i < count; i++) {
+ distances[i] = DBL_MAX;
+ flags[i] = false;
+ }
+
+ distances[source] = 0.0;
+ via[source] = source;
+
+ for (size_t i = 0; i < count - 1; i++) {
+ int32_t closest = closest_index(count, distances, flags);
+ flags[closest] = true;
+ for (size_t j = 0; j < count; j++) {
+ if ((!flags[j]) &&
+ (matrix[closest][j]) &&
+ (distances[closest] != DBL_MAX) &&
+ (distances[j] > distances[closest] + matrix[closest][j])) {
+ distances[j] = distances[closest] + matrix[closest][j];
+ via[j] = closest;
+ }
+ }
+ }
+
+ *eccent = 0;
+ for (size_t i = 0; i < count; i++) {
+ if (*eccent < distances[i]) {
+ *eccent = distances[i];
+ }
+ }
+
+ free(flags);
+}
+
+
+void main(int argc, char *argv[])
+{
+ double **distance_matrix;
+ double *shortest_distances;
+ int32_t *via_node;
+ int32_t node_count = DEFAULT_NODE_COUNT;
+ int32_t source_node = 0;
+ double node_eccentricity = 0.0;
+ double range_factor = 999.0 / (double)(RAND_MAX);
+ int32_t option;
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_node_count = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_node_count == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_node_count < MIN_NODE_COUNT) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be a "
+ "number less than %d.\n", MIN_NODE_COUNT);
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is too large */
+ if (user_node_count > MAX_NODE_COUNT) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be "
+ "more than %d.\n", MAX_NODE_COUNT);
+ exit(EXIT_FAILURE);
+ }
+ node_count = user_node_count;
+ } else {
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /* Allocate the memory space for all matrixes */
+ distance_matrix = (double **)malloc(node_count * sizeof(double *));
+ for (size_t i = 0; i < node_count; i++) {
+ distance_matrix[i] = (double *)malloc(node_count * sizeof(double));
+ }
+ shortest_distances = (double *)malloc(node_count * sizeof(double));
+ via_node = (int32_t *)malloc(node_count * sizeof(int32_t));
+
+ /* Initialize helper arrays and populate distance_matrix */
+ srand(1);
+ for (size_t i = 0; i < node_count; i++) {
+ shortest_distances[i] = 0.0;
+ via_node[i] = -1;
+ distance_matrix[i][i] = 0.0;
+ }
+ for (size_t i = 0; i < node_count; i++) {
+ for (size_t j = i + 1; j < node_count; j++) {
+ distance_matrix[i][j] = 1.0 + range_factor * (double)rand();
+ distance_matrix[j][i] = distance_matrix[i][j];
+ }
+ }
+
+ find_shortest_distances(shortest_distances, via_node, &node_eccentricity,
+ node_count, source_node, distance_matrix);
+
+ /* Control printing */
+ printf("CONTROL RESULT:\n");
+ printf(" Distance matrix (top left part):\n");
+ for (size_t i = 0; i < 3; i++) {
+ for (size_t j = 0; j < 3; j++) {
+ printf(" %7.2f", distance_matrix[i][j]);
+ }
+ printf("\n");
+ }
+ printf(" Source: %d (eccentricity: %f)\n",
+ source_node, node_eccentricity);
+ printf(" Destination Distance Via Node\n");
+ for (size_t i = 0; i < 3; i++) {
+ printf(" %5d %7.2f %4d\n",
+ i, shortest_distances[i], via_node[i]);
+ }
+
+ /* Free all previously allocated space */
+ for (size_t i = 0; i < node_count; i++) {
+ free(distance_matrix[i]);
+ }
+ free(distance_matrix);
+ free(shortest_distances);
+ free(via_node);
+}
diff --git a/tests/performance/nightly-tests/benchmarks/source/ dijkstra_int32/dijkstra_int32. c b/tests/performance/nightly- tests/benchmarks/source/ dijkstra_int32/dijkstra_int32. c
new file mode 100644
index 0000000000..2663cde943
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/ dijkstra_int32/dijkstra_int32. c
@@ -0,0 +1,192 @@
+/*
+ * Source file of a benchmark program involving calculations of the
+ * shortest distances between a source node and all other nodes in a
+ * graph of n nodes in which all nxn distances are defined as "int32".
+ * The number n can be given via command line, and the default is 2000.
+ * The algorithm used is Dijsktra's.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <limits.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of columns and rows in all matrixes*/
+#define DEFAULT_NODE_COUNT 2000
+#define MIN_NODE_COUNT 3
+#define MAX_NODE_COUNT 10000
+
+
+int32_t closest_index(int32_t count, int32_t *distances, bool *flags)
+{
+ int32_t closest;
+ int32_t minimum = INT_MAX;
+
+ for (size_t i = 0; i < count; i++) {
+ if (flags[i] == false && distances[i] <= minimum) {
+ closest = i;
+ minimum = distances[i];
+ }
+ }
+
+ return closest;
+}
+
+/**
+ * Calculate the shortest distances from the source node using Dijkstra method.
+ * @param (out) distances An array of shortest distances from the source node.
+ * @param (out) via An array of nodes needed to be taken as the the last
+ * before destination, for each destination.
+ * @param (out) eccent Eccentricity of the source node.
+ * @param (in) count The number of nodes.
+ * @param (in) source Source node.
+ * @param (in) matrix Distance matrix.
+ */
+void find_shortest_distances(int32_t *distances, int32_t *via, int32_t *eccent,
+ int32_t count, int32_t source, int32_t **matrix)
+{
+ bool *flags;
+
+ flags = (bool *)malloc(count * sizeof(bool));
+
+ for (size_t i = 0; i < count; i++) {
+ distances[i] = INT_MAX;
+ flags[i] = false;
+ }
+
+ distances[source] = 0;
+ via[source] = source;
+
+ for (size_t i = 0; i < count - 1; i++) {
+ int32_t closest = closest_index(count, distances, flags);
+ flags[closest] = true;
+ for (size_t j = 0; j < count; j++) {
+ if ((!flags[j]) &&
+ (matrix[closest][j]) &&
+ (distances[closest] != INT_MAX) &&
+ (distances[j] > distances[closest] + matrix[closest][j])) {
+ distances[j] = distances[closest] + matrix[closest][j];
+ via[j] = closest;
+ }
+ }
+ }
+
+ *eccent = 0;
+ for (size_t i = 0; i < count; i++) {
+ if (*eccent < distances[i]) {
+ *eccent = distances[i];
+ }
+ }
+
+ free(flags);
+}
+
+
+void main(int argc, char *argv[])
+{
+ int32_t **distance_matrix;
+ int32_t *shortest_distances;
+ int32_t *via_node;
+ int32_t node_count = DEFAULT_NODE_COUNT;
+ int32_t source_node = 0;
+ int32_t node_eccentricity = 0;
+ int32_t option;
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_node_count = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_node_count == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_node_count < MIN_NODE_COUNT) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be a "
+ "number less than %d.\n", MIN_NODE_COUNT);
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is too large */
+ if (user_node_count > MAX_NODE_COUNT) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be "
+ "more than %d.\n", MAX_NODE_COUNT);
+ exit(EXIT_FAILURE);
+ }
+ node_count = user_node_count;
+ } else {
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /* Allocate the memory space for all matrixes */
+ distance_matrix = (int32_t **)malloc(node_count * sizeof(int32_t *));
+ for (size_t i = 0; i < node_count; i++) {
+ distance_matrix[i] = (int32_t *)malloc(node_count * sizeof(int32_t));
+ }
+ shortest_distances = (int32_t *)malloc(node_count * sizeof(int32_t));
+ via_node = (int32_t *)malloc(node_count * sizeof(int32_t));
+
+ /* Initialize helper arrays and populate distance_matrix */
+ srand(1);
+ for (size_t i = 0; i < node_count; i++) {
+ shortest_distances[i] = 0;
+ via_node[i] = -1;
+ distance_matrix[i][i] = 0;
+ }
+ for (size_t i = 0; i < node_count; i++) {
+ for (size_t j = i + 1; j < node_count; j++) {
+ distance_matrix[i][j] = 1 + (rand()) / (RAND_MAX / 999);
+ distance_matrix[j][i] = distance_matrix[i][j];
+ }
+ }
+
+ find_shortest_distances(shortest_distances, via_node, &node_eccentricity,
+ node_count, source_node, distance_matrix);
+
+ /* Control printing */
+ printf("CONTROL RESULT:\n");
+ printf(" Distance matrix (top left part):\n");
+ for (size_t i = 0; i < 3; i++) {
+ for (size_t j = 0; j < 3; j++) {
+ printf(" %6d", distance_matrix[i][j]);
+ }
+ printf("\n");
+ }
+ printf(" Source: %d (eccentricity: %d)\n",
+ source_node, node_eccentricity);
+ printf(" Destination Distance Via Node\n");
+ for (size_t i = 0; i < 3; i++) {
+ printf(" %5d %3d %4d\n",
+ i, shortest_distances[i], via_node[i]);
+ }
+
+ /* Free all previously allocated space */
+ for (size_t i = 0; i < node_count; i++) {
+ free(distance_matrix[i]);
+ }
+ free(distance_matrix);
+ free(shortest_distances);
+ free(via_node);
+}
diff --git a/tests/performance/nightly-tests/benchmarks/source/ matmult_double/matmult_double. c b/tests/performance/nightly- tests/benchmarks/source/ matmult_double/matmult_double. c
new file mode 100644
index 0000000000..42bbb4717a
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/ matmult_double/matmult_double. c
@@ -0,0 +1,123 @@
+/*
+ * Source file of a benchmark program involving calculations of
+ * a product of two matrixes nxn whose elements are "double". The
+ * number n can be given via command line, and the default is 200.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of columns and rows in all matrixes*/
+#define DEFAULT_MATRIX_SIZE 200
+#define MIN_MATRIX_SIZE 2
+#define MAX_MATRIX_SIZE 200000
+
+void main(int argc, char *argv[])
+{
+ double **matrix_a;
+ double **matrix_b;
+ double **matrix_res;
+ size_t i;
+ size_t j;
+ size_t k;
+ int32_t matrix_size = DEFAULT_MATRIX_SIZE;
+ int32_t option;
+ double range_factor = 100.0 / (double)(RAND_MAX);
+
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_matrix_size = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_matrix_size == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_matrix_size < MIN_MATRIX_SIZE) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be a "
+ "number less than %d.\n", MIN_MATRIX_SIZE);
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is too large */
+ if (user_matrix_size > MAX_MATRIX_SIZE) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be "
+ "more than %d.\n", MAX_MATRIX_SIZE);
+ exit(EXIT_FAILURE);
+ }
+ matrix_size = user_matrix_size;
+ } else {
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /* Allocate the memory space for all matrixes */
+ matrix_a = (double **)malloc(matrix_size * sizeof(double *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_a[i] = (double *)malloc(matrix_size * sizeof(double));
+ }
+ matrix_b = (double **)malloc(matrix_size * sizeof(double *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_b[i] = (double *)malloc(matrix_size * sizeof(double));
+ }
+ matrix_res = (double **)malloc(matrix_size * sizeof(double *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_res[i] = (double *)malloc(matrix_size * sizeof(double));
+ }
+
+ /* Populate matrix_a and matrix_b with random numbers */
+ srand(1);
+ for (i = 0; i < matrix_size; i++) {
+ for (j = 0; j < matrix_size; j++) {
+ matrix_a[i][j] = range_factor * (double)rand();
+ matrix_b[i][j] = range_factor * (double)rand();
+ }
+ }
+
+ /* Calculate the product of two matrixes */
+ for (i = 0; i < matrix_size; i++) {
+ for (j = 0; j < matrix_size; j++) {
+ matrix_res[i][j] = 0.0;
+ for (k = 0; k < matrix_size; k++) {
+ matrix_res[i][j] += matrix_a[i][k] * matrix_b[k][j];
+ }
+ }
+ }
+
+ /* Control printing */
+ printf("CONTROL RESULT:\n");
+ printf(" %f %f\n", matrix_res[0][0], matrix_res[0][1]);
+ printf(" %f %f\n", matrix_res[1][0], matrix_res[1][1]);
+
+ /* Free all previously allocated space */
+ for (i = 0; i < matrix_size; i++) {
+ free(matrix_a[i]);
+ free(matrix_b[i]);
+ free(matrix_res[i]);
+ }
+ free(matrix_a);
+ free(matrix_b);
+ free(matrix_res);
+}
diff --git a/tests/performance/nightly-tests/benchmarks/source/ matmult_int32/matmult_int32.c b/tests/performance/nightly- tests/benchmarks/source/ matmult_int32/matmult_int32.c
new file mode 100644
index 0000000000..29a6eb000d
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/ matmult_int32/matmult_int32.c
@@ -0,0 +1,121 @@
+/*
+ * Source file of a benchmark program involving calculations of
+ * a product of two matrixes nxn whose elements are "int32_t". The
+ * number n can be given via command line, and the default is 200.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of columns and rows in all matrixes*/
+#define DEFAULT_MATRIX_SIZE 200
+#define MIN_MATRIX_SIZE 2
+#define MAX_MATRIX_SIZE 200000
+
+void main(int argc, char *argv[])
+{
+ int32_t **matrix_a;
+ int32_t **matrix_b;
+ int32_t **matrix_res;
+ size_t i;
+ size_t j;
+ size_t k;
+ int32_t matrix_size = DEFAULT_MATRIX_SIZE;
+ int32_t option;
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_matrix_size = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_matrix_size == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_matrix_size < MIN_MATRIX_SIZE) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be a "
+ "number less than %d.\n", MIN_MATRIX_SIZE);
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is too large */
+ if (user_matrix_size > MAX_MATRIX_SIZE) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be "
+ "more than %d.\n", MAX_MATRIX_SIZE);
+ exit(EXIT_FAILURE);
+ }
+ matrix_size = user_matrix_size;
+ } else {
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /* Allocate the memory space for all matrixes */
+ matrix_a = (int32_t **)malloc(matrix_size * sizeof(int32_t *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_a[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t));
+ }
+ matrix_b = (int32_t **)malloc(matrix_size * sizeof(int32_t *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_b[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t));
+ }
+ matrix_res = (int32_t **)malloc(matrix_size * sizeof(int32_t *));
+ for (i = 0; i < matrix_size; i++) {
+ matrix_res[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t));
+ }
+
+ /* Populate matrix_a and matrix_b with random numbers */
+ srand(1);
+ for (i = 0; i < matrix_size; i++) {
+ for (j = 0; j < matrix_size; j++) {
+ matrix_a[i][j] = (rand()) / (RAND_MAX / 100);
+ matrix_b[i][j] = (rand()) / (RAND_MAX / 100);
+ }
+ }
+
+ /* Calculate the product of two matrixes */
+ for (i = 0; i < matrix_size; i++) {
+ for (j = 0; j < matrix_size; j++) {
+ matrix_res[i][j] = 0;
+ for (k = 0; k < matrix_size; k++) {
+ matrix_res[i][j] += matrix_a[i][k] * matrix_b[k][j];
+ }
+ }
+ }
+
+ /* Control printing */
+ printf("CONTROL RESULT:\n");
+ printf(" %d %d\n", matrix_res[0][0], matrix_res[0][1]);
+ printf(" %d %d\n", matrix_res[1][0], matrix_res[1][1]);
+
+ /* Free all previously allocated space */
+ for (i = 0; i < matrix_size; i++) {
+ free(matrix_a[i]);
+ free(matrix_b[i]);
+ free(matrix_res[i]);
+ }
+ free(matrix_a);
+ free(matrix_b);
+ free(matrix_res);
+}
diff --git a/tests/performance/nightly-tests/benchmarks/source/qsort_ double/qsort_double.c b/tests/performance/nightly- tests/benchmarks/source/qsort_ double/qsort_double.c
new file mode 100644
index 0000000000..efc1b2eee1
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/qsort_ double/qsort_double.c
@@ -0,0 +1,104 @@
+/*
+ * Source file of a benchmark program involving sorting of an array
+ * of length n whose elements are "double". The default value for n
+ * is 300000, and it can be set via command line as well.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of elements in the array to be sorted */
+#define DEFAULT_ARRAY_LEN 300000
+#define MIN_ARRAY_LEN 3
+#define MAX_ARRAY_LEN 30000000
+
+/* Upper limit used for generation of random numbers */
+#define UPPER_LIMIT 1000.0
+
+/* Comparison function passed to qsort() */
+static int compare(const void *a, const void *b)
+{
+ if (*(const double *)a > *(const double *)b) {
+ return 1;
+ } else if (*(const double *)a < *(const double *)b) {
+ return -1;
+ }
+ return 0;
+}
+
+void main(int argc, char *argv[])
+{
+ double *array_to_be_sorted;
+ int32_t array_len = DEFAULT_ARRAY_LEN;
+ int32_t option;
+ double range_factor = UPPER_LIMIT / (double)(RAND_MAX);
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_array_len = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_array_len == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_array_len < MIN_ARRAY_LEN) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be a "
+ "number less than %d.\n", MIN_ARRAY_LEN);
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is too large */
+ if (user_array_len > MAX_ARRAY_LEN) {
+ fprintf(stderr, "Error ... Value for option '-n' cannot be "
+ "more than %d.\n", MAX_ARRAY_LEN);
+ exit(EXIT_FAILURE);
+ }
+ array_len = user_array_len;
+ } else {
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /* Allocate the memory space for the array */
+ array_to_be_sorted = (double *) malloc(array_len * sizeof(double));
+
+ /* Populate the_array with random numbers */
+ srand(1);
+ for (size_t i = 0; i < array_len; i++) {
+ array_to_be_sorted[i] = range_factor * (double)rand();
+ }
+
+ /* Sort the_array using qsort() */
+ qsort(array_to_be_sorted, array_len, sizeof(array_to_be_sorted[0]),
+ compare);
+
+ /* Control printing */
+ printf("CONTROL RESULT:\n");
+ printf("%14.10f %14.10f %14.10f\n",
+ array_to_be_sorted[0], array_to_be_sorted[1], array_to_be_sorted[2]);
+
+ /* Free all previously allocated space */
+ free(array_to_be_sorted);
+}
diff --git a/tests/performance/nightly-tests/benchmarks/source/qsort_ int32/qsort_int32.c b/tests/performance/nightly- tests/benchmarks/source/qsort_ int32/qsort_int32.c
new file mode 100644
index 0000000000..76ca9c3490
--- /dev/null
+++ b/tests/performance/nightly-tests/benchmarks/source/qsort_ int32/qsort_int32.c
@@ -0,0 +1,103 @@
+/*
+ * Source file of a benchmark program involving sorting of an array
+ * of length n whose elements are "int32_t". The default value for n
+ * is 300000, and it can be set via command line as well.
+ *
+ * This file is a part of the project "TCG Continuous Benchmarking".
+ *
+ * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
+ * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com >
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+/* Number of elements in the array to be sorted */
+#define DEFAULT_ARRAY_LEN 300000
+#define MIN_ARRAY_LEN 3
+#define MAX_ARRAY_LEN 30000000
+
+/* Upper limit used for generation of random numbers */
+#define UPPER_LIMIT 50000000
+
+/* Comparison function passed to qsort() */
+static int compare(const void *a, const void *b)
+{
+ if (*(const int32_t *)a > *(const int32_t *)b) {
+ return 1;
+ } else if (*(const int32_t *)a < *(const int32_t *)b) {
+ return -1;
+ }
+ return 0;
+}
+
+void main(int argc, char *argv[])
+{
+ int32_t *array_to_be_sorted;
+ int32_t array_len = DEFAULT_ARRAY_LEN;
+ int32_t option;
+
+ /* Parse command line options */
+ while ((option = getopt(argc, argv, "n:")) != -1) {
+ if (option == 'n') {
+ int32_t user_array_len = atoi(optarg);
+
+ /* Check if the value is a string or zero */
+ if (user_array_len == 0) {
+ fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
+ exit(EXIT_FAILURE);
+ }
+ /* Check if the value is a negative number */
+ if (user_array_len < MIN_ARRAY_LEN) {
+
| [Prev in Thread] | Current Thread | [Next in Thread] |