diff --git a/README.md b/README.md index 7ba11ca2fe..a605469de8 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ dependencies. Sandmark uses opam, with a static local repository, to build external libraries and applications. It then builds any sandmark OCaml benchmarks and any data dependencies. Following this it runs the -benchmarks as defined in the `run_config.json` +benchmarks as defined in the `sequential.json` These stages are implemented in: @@ -57,7 +57,7 @@ These stages are implemented in: - Runplan: the list of benchmarks which will run along with the measurement wrapper (e.g. orun or perf) is specified in - `run_config.json`. This config file is used to generate dune files + `sequential.json`. This config file is used to generate dune files which will run the benchmarks. - Build: dune is used to build all the sandmark OCaml benchmarks that @@ -65,7 +65,7 @@ These stages are implemented in: - Execute: dune is used to execute all the benchmarks sepcified in the runplan using the benchmark wrapper defined in - `run_config.json` and specified via the `RUN_BENCH_TARGET` variable + `sequential.json` and specified via the `RUN_BENCH_TARGET` variable passed to the makefile. ## Configuration of the compiler build @@ -140,8 +140,8 @@ sudo setcap cap_sys_nice=ep /usr/bin/chrt ### Configuring the benchmark runs -A config file can be specified with the environment variable `RUN_CONFIG_JSON`, -and the default value is `run_config.json`. This file lists the executable to +A config file can be specified with the environment variable `SEQUENTIAL_JSON`, +and the default value is `sequential.json`. This file lists the executable to run and the wrapper which will be used to collect data (e.g. orun or perf). You can edit this file to change benchmark parameters or wrappers. @@ -177,13 +177,13 @@ current tags are: The benchmarking machine `turing` is an Intel Xeon Gold 5120 CPU with 64GB of RAM housed at IITM. -The `run_config.json` file may be filtered based on the tag. For example, +The `sequential.json` file may be filtered based on the tag. For example, ```bash -$ TAG='"macro_bench"' make run_config_filtered.json +$ TAG='"macro_bench"' make sequential_filtered.json ``` -filters the `run_config.json` file to only contain the benchmarks tagged as +filters the `sequential.json` file to only contain the benchmarks tagged as `macro_bench`. ### Running benchmarks @@ -195,7 +195,7 @@ parallel benchmarks use `multibench_parallel`. You can also setup a custom bench and add only the benchmarks you care about. Sandmark has support to build and execute the serial benchmarks in -byte mode. A separate `run_config_byte.json` file has been created for +byte mode. A separate `sequential_byte.json` file has been created for the same. These benchmarks are relatively slower compared to their native execution. You can use the following commands to run the serial benchmarks in byte mode: @@ -203,7 +203,7 @@ benchmarks in byte mode: ```bash $ opam install dune.2.9.0 $ USE_SYS_DUNE_HACK=1 SANDMARK_CUSTOM_NAME=5.0.0 BUILD_BENCH_TARGET=bytebench \ - RUN_CONFIG_JSON=run_config_byte.json make ocaml-versions/5.0.0+stable.bench + SEQUENTIAL_JSON=sequential_byte.json make ocaml-versions/5.0.0+stable.bench ``` We can obtain throughput and latency results for the benchmarks. For obtaining @@ -261,17 +261,17 @@ You can add new benchmarks as follows: - **Add commands to run your applications:** Add an entry for your benchmark run to the appropriate config file; - `run_config.json` for sequential benchmarks and - `multicore_parallel_run_config.json` for parallel benchmarks. + `sequential.json` for sequential benchmarks and + `parallel_turing.json` for parallel benchmarks. ### Config files The `*_config.json` files used to build benchmarks - - **run_config.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing) - - **multicore_parallel_run_config.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing) - - **multicore_parallel_navajo_run_config.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine - - **micro_multicore.json** : To locally run multicore specific micro benchmarks + - **sequential.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing) + - **parallel_turing.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing) + - **parallel_navajo.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine + - **micro.json** : To locally run multicore specific micro benchmarks ### Benchmarks status @@ -360,7 +360,7 @@ work on OS X is to install GNU sed with homebrew and then update the | PIP_DEPENDENCIES | List of Python dependencies | ```intervaltree``` | building compiler and its dependencies | | PRE_BENCH_EXEC | Any specific commands that needed to be executed before the benchmark. For eg. `PRE_BENCH_EXEC='taskset --cpu-list 3 setarch uname -m --addr-no-randomize'` | null string | executing benchmark | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | | RUN_BENCH_TARGET | The executable to be used to run the benchmarks | `run_orun` | executing benchmark | -| RUN_CONFIG_JSON | Input file selection that contains the list of benchmarks | `run_config.json` | executing benchmark | +| SEQUENTIAL_JSON | Input file selection that contains the list of benchmarks | `sequential.json` | executing benchmark | | SANDMARK_DUNE_VERSION | Default dune version to be used | 2.9.0 | building compiler and its dependencies | | SANDMARK_OVERRIDE_PACKAGES | A list of dependency packages with versions that can be overrided (optional) | "" | building compiler and its dependencies | | SANDMARK_REMOVE_PACKAGES | A list of dependency packages to be dynamically removed (optional) | "" | building compiler and its dependencies | diff --git a/micro_multicore.json b/micro.json similarity index 100% rename from micro_multicore.json rename to micro.json diff --git a/multicore_parallel_navajo_run_config.json b/parallel_navajo.json similarity index 100% rename from multicore_parallel_navajo_run_config.json rename to parallel_navajo.json diff --git a/multicore_parallel_run_config.json b/parallel_turing.json similarity index 100% rename from multicore_parallel_run_config.json rename to parallel_turing.json diff --git a/multicore_parallel_run_config_filtered_filtered_2domains.json b/parallel_turing_filtered_filtered_2domains.json similarity index 100% rename from multicore_parallel_run_config_filtered_filtered_2domains.json rename to parallel_turing_filtered_filtered_2domains.json diff --git a/run_config.json b/sequential.json similarity index 100% rename from run_config.json rename to sequential.json diff --git a/run_config_byte.json b/sequential_byte.json similarity index 100% rename from run_config_byte.json rename to sequential_byte.json