SHA-256 (vs. Jolt)

As of May 2024

This note gives a benchmark comparison of proving execution of SHA-256 on a 32-byte input, on Valida and on Jolt. In this benchmark, single-core Valida proving was estimated to be about 6.4 times more efficient than multi-core Jolt proving, in terms of the time and energy expended on computing the proof. Multi-core Valida proving was estimated to be about 5.4 times faster than multi-core Jolt proving on this example. These are not the best results that could be obtained on this problem with Valida, since much work remains to be done on optimizing the prover and the code generated by the compiler.

Results

The raw data is available. The following table records the means, medians, and standard deviations of the various sample groups. All measurements are denoted in seconds.

Prover

Measure

Mean

Median

Standard deviation

Valida serial

User t.

1.1237

1.1235

0.01787

Valida parallel

User t.

2.7835

2.7865

0.06926

Jolt

User t.

7.1879

7.1945

0.2194

Valida serial

Wall clock t.

1.151

1.1505

0.0219

Valida parallel

Wall clock t.

0.249

0.2485

0.0051

Jolt

Wall clock t.

1.3565

1.357

0.0139

The following table displays the ratios between the mean measurements for Valida and Jolt of the user time and the wall clock time. A number greater than 1 indicates that Valida is faster; a number less than 1 indicates that Jolt is faster.

Measure

Valida condition

Valida advantage

User time

Serial

6.397

User time

Parallel

2.582

Wall clock time

Serial

1.179

Wall clock time

Parallel

5.443

Methodology

The following zk-VM versions were used:

The following version of the Valida LLVM compiler was used: 45bce621680189d5d006f88cbadbe9cbef403b89

The C implementation of SHA-256 which was compiled to run on Valida is a modified version of the reference implementation of SHA-256 by Brad Conte. That modified version is available here: https://github.com/lita-xyz/valida-c-examples/blob/main/sha256_32byte_in.c

The Rust implementation of SHA-256 which was compiled to run on Jolt is available in this repository.

The inputs for both Valida and Jolt are the same and are a 32-byte array of 5s.

The following commands were run to execute the benchmarks:

For Valida:

# excluded in the benchmarking time
./llvm-valida/build/bin/clang -c -target delendum ../../valida-c-examples/sha256_32byte_in.c -o ./buildValidaTests/sha256_32byte_in.o
./llvm-valida/build/bin/ld.lld --script=./llvm-valida/valida.ld -o ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.o

# included in the benchmarking time, run as a bash script and timed
RAYON_NUM_THREADS=32 ./valida/target/release/valida run ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.log
RAYON_NUM_THREADS=32 ./valida/target/release/valida prove ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.proof
RAYON_NUM_THREADS=32 ./valida/target/release/valida verify ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.proof

The above command shows multi-threaded execution; for single-threaded execution, 32 is replaced with 1.

Note that Valida currently fails to verify this program, but the output is checked to be correct by examining the log time. We are working on fixing this problem but the verification time should not make a meaningful impact on the results.

For Jolt, the following command was run from within the source repo, in the root directory:

git clone git@github.com:lita-xyz/benchmarks.git
cd sha256/jolt
time cargo run --release

In order to measure the run time of a program, this study used GNU time. The test system has a AMD Ryzen 9 7950X 16-Core Processor, with 32 threads, and 124.9 GB DDR5 RAM. During the tests there was no other programs running on the system other than the tests themselves. Tests were performed sequentially, one after another.

For some unknown reason, running cargo run --release on the test system caused the host program to be recompiled every time, which is not supposed to happen. This made it a little harder to measure the execution time of the Jolt prover. The build time listed in the output is therefore subtracted from the recorded time such that only the execution, proving, and verifying times are counted.

PreviousSHA-256 (vs. SP1)Nextzk-VM

Last updated