From 3d1f19c470d2e356a847e712a05b3ee2699f372f Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:16:39 +0530 Subject: [PATCH 1/8] Renamed the *run_config.json files --- README.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 7ba11ca2fe..cf478a5985 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ dependencies. Sandmark uses opam, with a static local repository, to build external libraries and applications. It then builds any sandmark OCaml benchmarks and any data dependencies. Following this it runs the -benchmarks as defined in the `run_config.json` +benchmarks as defined in the `sequential.json` These stages are implemented in: @@ -57,7 +57,7 @@ These stages are implemented in: - Runplan: the list of benchmarks which will run along with the measurement wrapper (e.g. orun or perf) is specified in - `run_config.json`. This config file is used to generate dune files + `sequential.json`. This config file is used to generate dune files which will run the benchmarks. - Build: dune is used to build all the sandmark OCaml benchmarks that @@ -65,7 +65,7 @@ These stages are implemented in: - Execute: dune is used to execute all the benchmarks sepcified in the runplan using the benchmark wrapper defined in - `run_config.json` and specified via the `RUN_BENCH_TARGET` variable + `sequential.json` and specified via the `RUN_BENCH_TARGET` variable passed to the makefile. ## Configuration of the compiler build @@ -140,8 +140,8 @@ sudo setcap cap_sys_nice=ep /usr/bin/chrt ### Configuring the benchmark runs -A config file can be specified with the environment variable `RUN_CONFIG_JSON`, -and the default value is `run_config.json`. This file lists the executable to +A config file can be specified with the environment variable `SEQUENTIAL_JSON`, +and the default value is `sequential.json`. This file lists the executable to run and the wrapper which will be used to collect data (e.g. orun or perf). You can edit this file to change benchmark parameters or wrappers. @@ -177,13 +177,13 @@ current tags are: The benchmarking machine `turing` is an Intel Xeon Gold 5120 CPU with 64GB of RAM housed at IITM. -The `run_config.json` file may be filtered based on the tag. For example, +The `sequential.json` file may be filtered based on the tag. For example, ```bash $ TAG='"macro_bench"' make run_config_filtered.json ``` -filters the `run_config.json` file to only contain the benchmarks tagged as +filters the `sequential.json` file to only contain the benchmarks tagged as `macro_bench`. ### Running benchmarks @@ -261,17 +261,17 @@ You can add new benchmarks as follows: - **Add commands to run your applications:** Add an entry for your benchmark run to the appropriate config file; - `run_config.json` for sequential benchmarks and - `multicore_parallel_run_config.json` for parallel benchmarks. + `sequential.json` for sequential benchmarks and + `parallel_turing.json` for parallel benchmarks. ### Config files The `*_config.json` files used to build benchmarks - - **run_config.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing) - - **multicore_parallel_run_config.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing) - - **multicore_parallel_navajo_run_config.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine - - **micro_multicore.json** : To locally run multicore specific micro benchmarks + - **sequential.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing) + - **parallel_turing.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing) + - **parallel_navajo.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine + - **micro.json** : To locally run multicore specific micro benchmarks ### Benchmarks status @@ -360,7 +360,7 @@ work on OS X is to install GNU sed with homebrew and then update the | PIP_DEPENDENCIES | List of Python dependencies | ```intervaltree``` | building compiler and its dependencies | | PRE_BENCH_EXEC | Any specific commands that needed to be executed before the benchmark. For eg. `PRE_BENCH_EXEC='taskset --cpu-list 3 setarch uname -m --addr-no-randomize'` | null string | executing benchmark | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | -| RUN_CONFIG_JSON | Input file selection that contains the list of benchmarks | `run_config.json` | executing benchmark | +| RUN_CONFIG_JSON | Input file selection that contains the list of benchmarks | `sequential.json` | executing benchmark | | SANDMARK_DUNE_VERSION | Default dune version to be used | 2.9.0 | building compiler and its dependencies | | SANDMARK_OVERRIDE_PACKAGES | A list of dependency packages with versions that can be overrided (optional) | "" | building compiler and its dependencies | | SANDMARK_REMOVE_PACKAGES | A list of dependency packages to be dynamically removed (optional) | "" | building compiler and its dependencies | From f0365094da684ad7e8407d2446c68ab49df0ef25 Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:19:21 +0530 Subject: [PATCH 2/8] Rename multicore_parallel_navajo_run_config.json to parallel_navajo.json --- multicore_parallel_navajo_run_config.json => parallel_navajo.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename multicore_parallel_navajo_run_config.json => parallel_navajo.json (100%) diff --git a/multicore_parallel_navajo_run_config.json b/parallel_navajo.json similarity index 100% rename from multicore_parallel_navajo_run_config.json rename to parallel_navajo.json From ed2f139e17a6f93d4f2a4b33a6c639c1ce992baa Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:20:03 +0530 Subject: [PATCH 3/8] Rename multicore_parallel_run_config.json to parallel_turing.json --- multicore_parallel_run_config.json => parallel_turing.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename multicore_parallel_run_config.json => parallel_turing.json (100%) diff --git a/multicore_parallel_run_config.json b/parallel_turing.json similarity index 100% rename from multicore_parallel_run_config.json rename to parallel_turing.json From edc9c05c60cb429c82679af4ac1759367d418193 Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:20:40 +0530 Subject: [PATCH 4/8] Rename multicore_parallel_run_config_filtered_filtered_2domains.json to parallel_turing_filtered_filtered_2domains.json --- ...omains.json => parallel_turing_filtered_filtered_2domains.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename multicore_parallel_run_config_filtered_filtered_2domains.json => parallel_turing_filtered_filtered_2domains.json (100%) diff --git a/multicore_parallel_run_config_filtered_filtered_2domains.json b/parallel_turing_filtered_filtered_2domains.json similarity index 100% rename from multicore_parallel_run_config_filtered_filtered_2domains.json rename to parallel_turing_filtered_filtered_2domains.json From ab33433fd28b2ea4b2f5936016d583a9cef0af68 Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:20:57 +0530 Subject: [PATCH 5/8] Rename run_config.json to sequential.json --- run_config.json => sequential.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename run_config.json => sequential.json (100%) diff --git a/run_config.json b/sequential.json similarity index 100% rename from run_config.json rename to sequential.json From 8982807ae13352c22b0e1d464753a983dea5a7f3 Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:21:16 +0530 Subject: [PATCH 6/8] Rename micro_multicore.json to micro.json --- micro_multicore.json => micro.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename micro_multicore.json => micro.json (100%) diff --git a/micro_multicore.json b/micro.json similarity index 100% rename from micro_multicore.json rename to micro.json From 34e7d358dca9ca300ea88f6673b445c6b1c83bf4 Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:23:56 +0530 Subject: [PATCH 7/8] Update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index cf478a5985..a605469de8 100644 --- a/README.md +++ b/README.md @@ -180,7 +180,7 @@ RAM housed at IITM. The `sequential.json` file may be filtered based on the tag. For example, ```bash -$ TAG='"macro_bench"' make run_config_filtered.json +$ TAG='"macro_bench"' make sequential_filtered.json ``` filters the `sequential.json` file to only contain the benchmarks tagged as @@ -195,7 +195,7 @@ parallel benchmarks use `multibench_parallel`. You can also setup a custom bench and add only the benchmarks you care about. Sandmark has support to build and execute the serial benchmarks in -byte mode. A separate `run_config_byte.json` file has been created for +byte mode. A separate `sequential_byte.json` file has been created for the same. These benchmarks are relatively slower compared to their native execution. You can use the following commands to run the serial benchmarks in byte mode: @@ -203,7 +203,7 @@ benchmarks in byte mode: ```bash $ opam install dune.2.9.0 $ USE_SYS_DUNE_HACK=1 SANDMARK_CUSTOM_NAME=5.0.0 BUILD_BENCH_TARGET=bytebench \ - RUN_CONFIG_JSON=run_config_byte.json make ocaml-versions/5.0.0+stable.bench + SEQUENTIAL_JSON=sequential_byte.json make ocaml-versions/5.0.0+stable.bench ``` We can obtain throughput and latency results for the benchmarks. For obtaining @@ -360,7 +360,7 @@ work on OS X is to install GNU sed with homebrew and then update the | PIP_DEPENDENCIES | List of Python dependencies | ```intervaltree``` | building compiler and its dependencies | | PRE_BENCH_EXEC | Any specific commands that needed to be executed before the benchmark. For eg. `PRE_BENCH_EXEC='taskset --cpu-list 3 setarch uname -m --addr-no-randomize'` | null string | executing benchmark | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | -| RUN_CONFIG_JSON | Input file selection that contains the list of benchmarks | `sequential.json` | executing benchmark | +| SEQUENTIAL_JSON | Input file selection that contains the list of benchmarks | `sequential.json` | executing benchmark | | SANDMARK_DUNE_VERSION | Default dune version to be used | 2.9.0 | building compiler and its dependencies | | SANDMARK_OVERRIDE_PACKAGES | A list of dependency packages with versions that can be overrided (optional) | "" | building compiler and its dependencies | | SANDMARK_REMOVE_PACKAGES | A list of dependency packages to be dynamically removed (optional) | "" | building compiler and its dependencies | From 3040e8c0ae1fc8309d70f51bc6a85700ea404ddf Mon Sep 17 00:00:00 2001 From: Kriti <75844962+itsIapetus@users.noreply.github.com> Date: Tue, 19 Apr 2022 22:32:29 +0530 Subject: [PATCH 8/8] Rename run_config_byte.json to sequential_byte.json --- run_config_byte.json => sequential_byte.json | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename run_config_byte.json => sequential_byte.json (100%) diff --git a/run_config_byte.json b/sequential_byte.json similarity index 100% rename from run_config_byte.json rename to sequential_byte.json