Skip to content

Control benchmarks from the commandline #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Daeinar merged 6 commits into from
Nov 5, 2025
Merged

Control benchmarks from the commandline #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7837130
feat: control benchmarks from the commandline
Daeinar Nov 4, 2025
bae89d6
feat: control more benchmarks from the commandline
Daeinar Nov 4, 2025
c9983f9
feat: auctioneer benchmarks selection code
Daeinar Nov 4, 2025
2988dfa
fix: code cleanup and benchmarking instructions
Daeinar Nov 5, 2025
8cf6749
fix: readme formatting
Daeinar Nov 5, 2025
b73514c
...
Daeinar Nov 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
table1a:
N=128 cargo bench --bench veceq
N=1024 cargo bench --bench veceq
N=8192 cargo bench --bench veceq
N=128 cargo bench --bench nonzero
N=1024 cargo bench --bench nonzero
N=8192 cargo bench --bench nonzero
N=128 cargo bench --bench lderivative
N=1024 cargo bench --bench lderivative
N=8192 cargo bench --bench lderivative
N=128 cargo bench --bench ipa
N=1024 cargo bench --bench ipa
N=8192 cargo bench --bench ipa

table1b:
M=32 N=128 cargo bench --bench auctioneer_r1
M=32 N=1024 cargo bench --bench auctioneer_r1
M=32 N=8192 cargo bench --bench auctioneer_r1
M=128 N=128 cargo bench --bench auctioneer_r1
M=128 N=1024 cargo bench --bench auctioneer_r1
M=128 N=8192 cargo bench --bench auctioneer_r1
M=256 N=128 cargo bench --bench auctioneer_r1
M=256 N=1024 cargo bench --bench auctioneer_r1
M=256 N=8192 cargo bench --bench auctioneer_r1

table1c:
M=32 N=128 cargo bench --bench auctioneer_r2
M=32 N=1024 cargo bench --bench auctioneer_r2
M=32 N=8192 cargo bench --bench auctioneer_r2
M=128 N=128 cargo bench --bench auctioneer_r2
M=128 N=1024 cargo bench --bench auctioneer_r2
M=128 N=8192 cargo bench --bench auctioneer_r2
M=256 N=128 cargo bench --bench auctioneer_r2
M=256 N=1024 cargo bench --bench auctioneer_r2
M=256 N=8192 cargo bench --bench auctioneer_r2
102 changes: 64 additions & 38 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,67 @@ To setup Rust, please follow the [official installation instructions](https://ww

## Benchmarks

There are six microbenchmarks, namely for the computation of the four validity proofs, the AV protocol, and the results vector:

- **Benchmark 1: Validity proof $\pi_{x_i}$ (Table 1a)**
```
cargo bench --bench veceq
```

- **Benchmark 2: Validity proof $\pi_{r_i}$ (Table 1a)**
```
cargo bench --bench nonzero
```

- **Benchmark 3: Validity proof $\pi_{b_i}$ (Table 1a)**
```
cargo bench --bench lderivative
```

- **Benchmark 4: Validity proof $\pi_{Z_i}$ (Table 1a)**
```
cargo bench --bench ipa
```

- **Benchmark 5: AV matrix $Y$ (Table 1b)**
```
cargo bench --bench auctioneer_r1
```

- **Benchmark 6: Results vector $R$ (Table 1c)**
```
cargo bench --bench auctioneer_r2
```







There are six different microbenchmarks in total, namely for the computation of the four validity proofs, the AV protocol, and the results vector.
To reproduce the benchmarks presented in Table 1, simply execute:

```
make table1a
make table1b
make table1c
```

For more fine-grained benchmarking, consider the following instructions.

**Benchmark 1: Validity proof $\pi_{x_i}$ (Table 1a)**
```
N=128 cargo bench --bench veceq
N=1024 cargo bench --bench veceq
N=8192 cargo bench --bench veceq
```

**Benchmark 2: Validity proof $\pi_{r_i}$ (Table 1a)**
```
N=128 cargo bench --bench nonzero
N=1024 cargo bench --bench nonzero
N=8192 cargo bench --bench nonzero
```

**Benchmark 3: Validity proof $\pi_{b_i}$ (Table 1a)**
```
N=128 cargo bench --bench lderivative
N=1024 cargo bench --bench lderivative
N=8192 cargo bench --bench lderivative
```

**Benchmark 4: Validity proof $\pi_{Z_i}$ (Table 1a)**
```
N=128 cargo bench --bench ipa
N=1024 cargo bench --bench ipa
N=8192 cargo bench --bench ipa
```

**Benchmark 5: AV matrix $Y$ (Table 1b)**
```
M=32 N=128 cargo bench --bench auctioneer_r1
M=32 N=1024 cargo bench --bench auctioneer_r1
M=32 N=8192 cargo bench --bench auctioneer_r1
M=128 N=128 cargo bench --bench auctioneer_r1
M=128 N=1024 cargo bench --bench auctioneer_r1
M=128 N=8192 cargo bench --bench auctioneer_r1
M=256 N=128 cargo bench --bench auctioneer_r1
M=256 N=1024 cargo bench --bench auctioneer_r1
M=256 N=8192 cargo bench --bench auctioneer_r1
```

**Benchmark 6: Results vector $R$ (Table 1c)**
```
M=32 N=128 cargo bench --bench auctioneer_r2
M=32 N=1024 cargo bench --bench auctioneer_r2
M=32 N=8192 cargo bench --bench auctioneer_r2
M=128 N=128 cargo bench --bench auctioneer_r2
M=128 N=1024 cargo bench --bench auctioneer_r2
M=128 N=8192 cargo bench --bench auctioneer_r2
M=256 N=128 cargo bench --bench auctioneer_r2
M=256 N=1024 cargo bench --bench auctioneer_r2
M=256 N=8192 cargo bench --bench auctioneer_r2
```
100 changes: 29 additions & 71 deletions benches/auctioneer_r1.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ use ark_ff::Zero;
use ark_std::{test_rng, UniformRand};
use cipher_bazaar::auctioneer::Auctioneer;
use criterion::{criterion_group, criterion_main, Criterion};
use std::env;

/* RUN WITH: cargo bench --bench auctioneer_r1 */
/* RUN WITH: M=32 N=128 cargo bench --bench auctioneer_r1 */

fn setup_round_1<const N: usize, const B: usize>() -> Auctioneer<N, B, G1Projective> {
let mut rng = test_rng();
Expand Down Expand Up @@ -39,87 +40,44 @@ fn setup_round_1<const N: usize, const B: usize>() -> Auctioneer<N, B, G1Project
a
}

fn setup_round_2<const N: usize, const B: usize>() -> Auctioneer<N, B, G1Projective> {
let mut rng = test_rng();
let g = G1Projective::generator();

let mut a = Auctioneer::<N, B, G1Projective>::new();
let mut secrets = vec![vec![F::zero(); N]; B];
let mut first_msgs = vec![vec![G1Affine::zero(); N]; B];

// initialize n msgs fro each party
for i in 0..B {
for j in 0..N {
secrets[i][j] = F::rand(&mut rng);
}
}

// initialize n msgs fro each party
for i in 0..B {
for j in 0..N {
first_msgs[i][j] = g.mul(secrets[i][j]).into();
}
}

// each party sends it's first round msgs
for i in 0..B {
a.register_msgs(&first_msgs[i], i).unwrap();
}

// we get output for each party per round
// where each row is of len B (output of av for each party)
let fr_result = a.output_first_round();

let mut second_msgs = vec![vec![G1Affine::zero(); N]; B];
for i in 0..B {
for j in 0..N {
second_msgs[i][j] = fr_result[j][i].mul(secrets[i][j]).into();
}
}

// each party sends it's second round msgs
for i in 0..B {
a.register_msgs(&second_msgs[i], i).unwrap();
}

a
}

fn bench_second_round<const N: usize, const B: usize>(
a: Auctioneer<N, B, G1Projective>,
) -> Vec<G1Affine> {
let mut a_clone = a.clone();
a_clone.output_second_round()
}

fn bench_first_round<const N: usize, const B: usize>(
a: Auctioneer<N, B, G1Projective>,
) -> Vec<Vec<G1Affine>> {
let mut a_clone = a.clone();
a_clone.output_first_round()
}

fn round_1(c: &mut Criterion) {
const N: usize = 8192;
const B: usize = 256;

let a = setup_round_1::<N, B>();
let id = format!("Round1: range = {}, bidders = {}", N, B);
fn round_1<const M: usize, const N: usize>(c: &mut Criterion) {
let a = setup_round_1::<N, M>();
let id = format!("Round1: range = {}, bidders = {}", N, M);
c.bench_function(&id, |b| b.iter(|| bench_first_round(a.clone())));
}

fn round_2(c: &mut Criterion) {
const N: usize = 32;
const B: usize = 32;

let a = setup_round_2::<N, B>();
let id = format!("Round2: range = {}, bidders = {}", N, B);
c.bench_function(&id, |b| b.iter(|| bench_second_round(a.clone())));
}

fn criterion_benchmark(c: &mut Criterion) {
round_1(c);
// round_2(c);
// bidders
let m: usize = env::var("M")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(32); // default value

// range
let n: usize = env::var("N")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(128); // default value

match (m, n) {
(32, 128) => round_1::<32, 128>(c),
(32, 1024) => round_1::<32, 1024>(c),
(32, 8192) => round_1::<32, 8192>(c),
(128, 128) => round_1::<128, 128>(c),
(128, 1024) => round_1::<128, 1024>(c),
(128, 8192) => round_1::<128, 8192>(c),
(256, 128) => round_1::<256, 128>(c),
(256, 1024) => round_1::<256, 1024>(c),
(256, 8192) => round_1::<256, 8192>(c),
_ => panic!("Unsupported parameter combination (M, N): ({}, {})", m, n),
}
}

criterion_group!(benches, criterion_benchmark);
Expand Down
84 changes: 29 additions & 55 deletions benches/auctioneer_r2.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,38 +6,9 @@ use ark_ff::Zero;
use ark_std::{test_rng, UniformRand};
use cipher_bazaar::auctioneer::Auctioneer;
use criterion::{criterion_group, criterion_main, Criterion};
use std::env;

/* RUN WITH: cargo bench --bench auctioneer_r2 */

fn setup_round_1<const N: usize, const B: usize>() -> Auctioneer<N, B, G1Projective> {
let mut rng = test_rng();
let g = G1Projective::generator();

let mut a = Auctioneer::<N, B, G1Projective>::new();
let mut secrets = vec![vec![F::zero(); N]; B];
let mut first_msgs = vec![vec![G1Affine::zero(); N]; B];

// initialize n msgs fro each party
for i in 0..B {
for j in 0..N {
secrets[i][j] = F::rand(&mut rng);
}
}

// initialize n msgs fro each party
for i in 0..B {
for j in 0..N {
first_msgs[i][j] = g.mul(secrets[i][j]).into();
}
}

// each party sends it's first round msgs
for i in 0..B {
a.register_msgs(&first_msgs[i], i).unwrap();
}

a
}
/* RUN WITH: M=32 N=1024 cargo bench --bench auctioneer_r2 */

fn setup_round_2<const N: usize, const B: usize>() -> Auctioneer<N, B, G1Projective> {
let mut rng = test_rng();
Expand Down Expand Up @@ -92,34 +63,37 @@ fn bench_second_round<const N: usize, const B: usize>(
a_clone.output_second_round()
}

fn bench_first_round<const N: usize, const B: usize>(
a: Auctioneer<N, B, G1Projective>,
) -> Vec<Vec<G1Affine>> {
let mut a_clone = a.clone();
a_clone.output_first_round()
}

fn round_1(c: &mut Criterion) {
const N: usize = 8192;
const B: usize = 256;

let a = setup_round_1::<N, B>();
let id = format!("Round1: range = {}, bidders = {}", N, B);
c.bench_function(&id, |b| b.iter(|| bench_first_round(a.clone())));
}

fn round_2(c: &mut Criterion) {
const N: usize = 8192;
const B: usize = 256;

let a = setup_round_2::<N, B>();
let id = format!("Round2: range = {}, bidders = {}", N, B);
fn round_2<const M: usize, const N: usize>(c: &mut Criterion) {
let a = setup_round_2::<N, M>();
let id = format!("Round2: range = {}, bidders = {}", N, M);
c.bench_function(&id, |b| b.iter(|| bench_second_round(a.clone())));
}

fn criterion_benchmark(c: &mut Criterion) {
// round_1(c);
round_2(c);
// bidders
let m: usize = env::var("M")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(32); // default value

// range
let n: usize = env::var("N")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(128); // default value

match (m, n) {
(32, 128) => round_2::<32, 128>(c),
(32, 1024) => round_2::<32, 1024>(c),
(32, 8192) => round_2::<32, 8192>(c),
(128, 128) => round_2::<128, 128>(c),
(128, 1024) => round_2::<128, 1024>(c),
(128, 8192) => round_2::<128, 8192>(c),
(256, 128) => round_2::<256, 128>(c),
(256, 1024) => round_2::<256, 1024>(c),
(256, 8192) => round_2::<256, 8192>(c),
_ => panic!("Unsupported parameter combination (M, N): ({}, {})", m, n),
}
}

criterion_group!(benches, criterion_benchmark);
Expand Down
Loading