SHA-256 (vs. RISC Zero)

As of May 2024

This note gives a benchmark comparison of proving execution of SHA-256 on a 32-byte input, on Valida and on RISC Zero. In this benchmark, single-core Valida proving was estimated to be about 65.8 times more efficient than multi-core RISC Zero proving, in terms of the time and energy expended on computing the proof. Multi-core Valida proving was estimated to be about 11.6 times faster than multi-core RISC Zero proving on this example. These are not the best results that could be obtained on this problem with Valida, since much work remains to be done on optimizing the prover and the code generated by the compiler.

Results

The raw data is available. The following table records the means, medians, and standard deviations of the various sample groups. All measurements are denoted in seconds.

Prover

Measure

Mean

Median

Standard deviation

Valida serial

User t.

1.1237

1.1235

0.01787

Valida parallel

User t.

2.7835

2.7865

0.06926

RISC Zero

User t.

73.989

73.995

0.1083

Valida serial

Wall clock t.

1.151

1.1505

0.0219

Valida parallel

Wall clock t.

0.249

0.2485

0.0051

RISC Zero

Wall clock t.

2.901

2.902

0.00727

The following table displays the ratios between the mean measurements for Valida and RISC Zero of the user time and the wall clock time. A number greater than 1 indicates that Valida is faster; a number less than 1 indicates that RISC Zero is faster.

Measure

Valida condition

Valida advantage

User time

Serial

65.84

User time

Parallel

26.58

Wall clock time

Serial

2.52

Wall clock time

Parallel

11.64

Methodology

The following zk-VM versions were used:

The following version of the Valida LLVM compiler was used: 45bce621680189d5d006f88cbadbe9cbef403b89

The C implementation of SHA-256 which was compiled to run on Valida is a modified version of the reference implementation of SHA-256 by Brad Conte. That modified version is available here: https://github.com/lita-xyz/valida-c-examples/blob/main/sha256_32byte_in.c

The Rust implementation of SHA-256 which was compiled to run on RISC Zero was a modification of their sha example. Specially, examples/sha/src/main.rs was modified to the following to give it a 32 byte fixed input:

// Copyright 2024 RISC Zero, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use risc0_zkvm::{default_prover, sha::Digest, ExecutorEnv, Receipt};
use sha_methods::{HASH_ELF, HASH_ID, HASH_RUST_CRYPTO_ELF};

/// Hash the given bytes, returning the digest and a [Receipt] that can
/// be used to verify that the that the hash was computed correctly (i.e. that
/// the Prover knows a preimage for the given SHA-256 hash)
///
/// Select which method to use with `use_rust_crypto`.
/// HASH_ELF uses the risc0_zkvm::sha interface for hashing.
/// HASH_RUST_CRYPTO_ELF uses RustCrypto's [sha2] crate, patched to use the RISC
/// Zero accelerator. See `src/methods/guest/Cargo.toml` for the patch
/// definition, which can be used to enable SHA-256 accelerrator support
/// everywhere the [sha2] crate is used.
fn provably_hash(input: &str, use_rust_crypto: bool) -> (Digest, Receipt) {
    let env = ExecutorEnv::builder()
        .write(&input)
        .unwrap()
        .build()
        .unwrap();

    let elf = if use_rust_crypto {
        HASH_RUST_CRYPTO_ELF
    } else {
        HASH_ELF
    };

    // Obtain the default prover.
    let prover = default_prover();

    // Produce a receipt by proving the specified ELF binary.
    let receipt = prover.prove(env, elf).unwrap();

    let digest = receipt.journal.decode().unwrap();
    (digest, receipt)
}

fn main() {
    // Ensure the input is the same as the other programs for benchmarking
    let input = "55555555555555555555555555555555";

    // Prove hash the message.
    let (digest, receipt) = provably_hash(&input, false);

    // Verify the receipt, ensuring the prover knows a valid SHA-256 preimage.
    receipt
        .verify(HASH_ID)
        .expect("receipt verification failed");

    println!("I provably know data whose SHA-256 hash is {}", digest);
}

#[cfg(test)]
mod tests {
    use serial_test::serial;
    use sha_methods::{HASH_ID, HASH_RUST_CRYPTO_ID};

    #[test]
    #[serial]
    fn hash_abc() {
        let (digest, receipt) = super::provably_hash("abc", false);
        receipt.verify(HASH_ID).unwrap();
        assert_eq!(
            hex::encode(digest.as_bytes()),
            "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
            "We expect to match the reference SHA-256 hash of the standard test value 'abc'"
        );
    }

    #[test]
    #[serial]
    fn hash_abc_rust_crypto() {
        let (digest, receipt) = super::provably_hash("abc", true);
        receipt.verify(HASH_RUST_CRYPTO_ID).unwrap();
        assert_eq!(
            hex::encode(digest.as_bytes()),
            "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
            "We expect to match the reference SHA-256 hash of the standard test value 'abc'"
        );
    }
}

The inputs for both Valida and RISC Zero are 32-bytes.

The following commands were run to execute the benchmarks:

For Valida:

# excluded in the benchmarking time
./llvm-valida/build/bin/clang -c -target delendum ../../valida-c-examples/sha256_32byte_in.c -o ./buildValidaTests/sha256_32byte_in.o
./llvm-valida/build/bin/ld.lld --script=./llvm-valida/valida.ld -o ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.o

# included in the benchmarking time, run as a bash script and timed
RAYON_NUM_THREADS=32 ./valida/target/release/valida run ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.log
RAYON_NUM_THREADS=32 ./valida/target/release/valida prove ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.proof
RAYON_NUM_THREADS=32 ./valida/target/release/valida verify ./buildValidaTests/sha256_32byte_in.out ./buildValidaTests/sha256_32byte_in.proof

The above command shows multi-threaded execution; for single-threaded execution, 32 is replaced with 1.

Note that Valida currently fails to verify this program, but the output is checked to be correct by examining the log time. We are working on fixing this problem but the verification time should not make a meaningful impact on the results.

For RISC Zero, first, make sure you are in the correct branch by following the README instruction and switch to release version 2.1. Then, the following command was run from within the source repo, in the examples/sha/ directory:

time cargo run --release

In order to measure the run time of a program, this study used GNU time. The test system has a AMD Ryzen 9 7950X 16-Core Processor, with 32 threads, and 124.9 GB DDR5 RAM. During the tests there was no other programs running on the system other than the tests themselves. Tests were performed sequentially, one after another.

For some unknown reason, running cargo run --release on the test system caused the host program to be recompiled every time, which is not supposed to happen. This made it a little harder to measure the execution time of the RISC Zero prover. The build time listed in the output is therefore subtracted from the recorded time such that only the execution, proving, and verifying times are counted.

PreviousFibonacci (Rust vs. C)NextSHA-256 (vs. SP1)

Last updated