How to Design a Private A/B Testing Framework Using ZK-Proofs

Traditional A/B testing platforms like Optimizely or Google Optimize require collecting granular user interaction data, creating significant privacy risks and compliance burdens. A privacy-preserving A/B testing framework uses cryptographic techniques, primarily zero-knowledge proofs (ZKPs), to allow analysts to compute aggregate statistics—such as conversion rates or average session duration—while keeping each individual user's data encrypted and private. This approach is critical for Web3 applications, healthcare tech, or any context handling sensitive information, enabling data-driven decisions without violating user trust or regulations like GDPR.

The core architectural shift involves moving computation to the client side. Instead of sending raw event logs to a central server, each client device (e.g., a user's browser or wallet) locally computes a cryptographic commitment to their assigned variant (A or B) and their outcome metric. For a conversion test, this could be a Pedersen commitment to a tuple (variant, converted), where converted is 0 or 1. These commitments are then submitted to a public ledger or a designated collector. The individual data points remain hidden, but the commitments are binding and hiding.

To analyze the experiment, a verifier (or a smart contract) uses a ZK-SNARK circuit to validate the aggregate result. The circuit takes the list of commitments as a public input and a private witness containing the opening of each commitment. It proves, for example, that "the average conversion rate for variant B is 12.5%, and this was calculated correctly from 1,000 valid user commitments, without revealing any single user's variant assignment or outcome." Frameworks like Circom or Halo2 are used to write this circuit logic. The output is a succinct proof that can be verified on-chain.

Implementing this requires careful design of the ZKP circuit. Key circuit constraints must ensure:

• Each commitment is properly formed and corresponds to a valid variant (e.g., 0 or 1).
• Outcome values are within a valid range (e.g., 0 or 1 for conversion, or a bounded integer for engagement time).
• The computation of the aggregate statistic (mean, sum, standard deviation) is performed correctly over the revealed private values.
• No other information about the private inputs is leaked. A sample Circom template for a sum check might enforce that the sum of revealed outcomes equals the publicly claimed total.

For on-chain verification and incentivization, the framework can be integrated with a smart contract. The contract stores the list of commitment hashes during the experiment phase. After the test concludes, an analyst submits the final aggregate result along with a ZK-SNARK proof. The contract verifies the proof against the stored commitments. If valid, it accepts the result, which can then trigger actions like deploying a new UI variant. This creates a trust-minimized, transparent, and private testing pipeline. Projects like Semaphore demonstrate similar patterns for anonymous signaling.

While powerful, this design introduces challenges: ZKP generation has computational overhead, requiring efficient client-side libraries (like SnarkJS in-browser). The system must also guard against Sybil attacks—ensuring one user doesn't submit multiple commitments. This can be mitigated by tying commitments to a provable unique identity, such as a ZK proof of membership in a verified group or a proof of ownership of a non-fungible token (NFT). Despite these complexities, the privacy guarantees make it a compelling architecture for the future of ethical experimentation.

A strong background in cryptography and blockchain development is essential. You should be comfortable with zero-knowledge proof concepts like zk-SNARKs and zk-STARKs, understanding their proving and verification mechanisms. Familiarity with cryptographic primitives—hash functions, digital signatures, and commitment schemes—is also required. On the blockchain side, experience with smart contract development on EVM-compatible chains (like Ethereum or Polygon) is necessary for deploying the verification contracts. Knowledge of a frontend framework like React or Vue.js is recommended for building the user-facing dashboard.

For the development stack, you will need a ZK-specific proving system. We recommend using Circom 2 for writing arithmetic circuits and SnarkJS for generating and verifying proofs, as they are well-documented and widely adopted. For on-chain verification, you'll use a Solidity library like the Verifier.sol contract generated by SnarkJS. The backend, which handles user grouping and proof aggregation, can be built with Node.js or Python. Essential tools include Node.js v18+, a package manager like npm or yarn, and the Circom compiler. You will also need access to an RPC endpoint for a testnet like Sepolia or Mumbai for deployment.

Your system must meet specific hardware requirements due to the computational intensity of proof generation. We recommend a machine with a multi-core processor (Intel i7 or equivalent AMD Ryzen) and at least 16 GB of RAM. Proof generation, especially for circuits with many constraints, can be memory-intensive. You will also need a minimum of 20 GB of free storage for dependencies, compiled circuits, and project files. A stable internet connection is required for fetching packages and interacting with blockchain networks. For optimal performance in a production setting, consider using cloud servers with scalable compute resources.

Traditional A/B testing platforms, like Optimizely or Google Optimize, require collecting granular user data—clicks, conversions, demographics—to compute statistical significance. This creates privacy risks and compliance burdens. A ZK-powered A/B testing framework inverts this model. Instead of sending raw data to a central server, computation happens on the client or a trusted enclave. The system only publishes a zero-knowledge proof attesting that the analysis (e.g., a t-test calculation) was performed correctly on valid, unseen data. This allows experimenters to verify result integrity without accessing the underlying user information, aligning with principles of data minimization and regulatory frameworks like GDPR.

The core system architecture consists of three main layers. The Client-Side SDK is embedded in the application (web or mobile) and is responsible for randomly assigning users to control (A) or treatment (B) groups using a verifiable random function (VRF). It locally records key metrics like conversions. The Proving Circuit, typically written in a ZK domain-specific language like Circom or Noir, encodes the statistical test logic. This circuit takes private inputs (the user's group assignment and outcome) and public inputs (the aggregated experiment parameters) to generate a proof. Finally, a Verification & Aggregation Service on-chain or off-chain collects proofs from many users, verifies each one, and aggregates the results to compute the final experiment outcome.

A critical design choice is the data model and proof structure. For a binomial metric like click-through rate, each user's data can be represented as a private tuple: (group_assignment, did_convert). The ZK circuit's public output could be the sums: total users in group A, conversions in group A, total users in group B, conversions in group B. The circuit logic ensures these sums are correctly computed from the private inputs without revealing any individual's data. For more complex metrics like average revenue per user (ARPU), the circuit would need to securely compute sums of continuous values, requiring careful attention to fixed-point arithmetic within the ZK proof system.

Implementing this requires selecting a ZK proof system. zk-SNARKs (e.g., via Circom and the Groth16 prover) offer small proof sizes and fast verification, ideal for on-chain settlement, but require a trusted setup. zk-STARKs (e.g., with Starky) are transparent (no trusted setup) and faster to generate, but have larger proof sizes. For a high-throughput A/B testing platform expecting millions of user proofs, the proving performance and cost are paramount. Often, a hybrid approach is used: proofs are generated off-chain in a distributed proving service and only the final aggregated result—or a single validity proof for the aggregation—is posted to a blockchain like Ethereum or a Layer 2 (L2) such as Arbitrum for immutable, verifiable record-keeping.

The final architectural component is the oracle or relay that bridges off-chain data availability with on-chain verification. Users' encrypted data or data commitments may be stored in a decentralized storage network like IPFS or Arweave, with the content identifier (CID) referenced in the proof. The verifier contract can then confirm the proof is valid for the given CID. This creates a full trustless pipeline: from client-side data collection to verifiable, privacy-preserving statistical analysis. Frameworks like Semaphore or zkKit can serve as foundational libraries for managing group identities and anonymous signaling within this architecture.

In practice, designing this system involves trade-offs between privacy, cost, and usability. While fully on-chain verification provides maximum trustlessness, the gas costs for verifying individual user proofs can be prohibitive. Batching proofs using schemes like Plonk or recursive SNARKs can aggregate thousands of user actions into a single verification step. Furthermore, the client-side proving must be efficient enough not to degrade user experience; WebAssembly-based provers or dedicated co-processors may be necessary. This architecture represents a significant shift towards verifiable computation as a service, enabling businesses to conduct rigorous experimentation while upholding a strong standard of user privacy.

The first step in a private A/B testing framework is to assign users to a test group (e.g., A or B) in a way that is both random and verifiable, without revealing the assignment logic or the user's identity. Traditional methods rely on a trusted server, creating a single point of failure and opacity. Using zero-knowledge proofs (ZKPs), we can decentralize this trust. A smart contract can act as a public source of randomness, like a verifiable random function (VRF) from Chainlink or a commit-reveal scheme, to generate a seed. A user's deterministic group assignment is then computed off-chain using this seed and a public identifier (like a hashed user ID).

The core innovation is proving the correctness of this assignment without disclosure. A user, or a designated prover, generates a ZK-SNARK or ZK-STARK proof. This proof cryptographically attests that: 1) The user's public input (hashed ID) was used correctly, 2) The assignment algorithm (e.g., group = hash(seed, userHash) % 2) was followed, and 3) The resulting group (0 or 1) matches the claimed assignment. The proof does not reveal the seed, the raw user ID, or the internal computation steps. This proof can be verified by anyone, including the smart contract managing the test, ensuring the randomization was fair.

For developers, implementing this requires a circuit written in a ZK domain-specific language like Circom or Noir. The circuit's public inputs are the commitment to the random seed and the user's public identifier hash. The private inputs are the actual seed and the user's pre-image. The output is the group index. After compiling the circuit and generating a proving key, users can generate proofs client-side. Here's a conceptual Circom template:

circomCopy
template GroupAssignment(nBits) {
    signal input seed; // Private
    signal input userIdPreimage; // Private
    signal input seedCommitment; // Public
    signal input userIdHash; // Public
    signal output group;

    // Verify inputs match commitments
    component hashSeed = Poseidon(nBits);
    hashSeed.in <== seed;
    seedCommitment === hashSeed.out;

    component hashUser = Poseidon(nBits);
    hashUser.in <== userIdPreimage;
    userIdHash === hashUser.out;

    // Compute deterministic group
    component mix = Poseidon(nBits);
    mix.in[0] <== seed;
    mix.in[1] <== userIdPreimage;
    group <== mix.out % 2;
}

This approach has critical benefits for A/B testing. Experiment integrity is guaranteed because the assignment logic is fixed in the circuit code and the random seed is publicly committed to before the experiment begins, preventing cherry-picking. User privacy is maintained as the proof reveals only the group assignment, not the user's identity or behavior. Universal verifiability allows any third party to audit the fairness of the group allocation by verifying the ZK proofs against the public commitments, fostering trust without requiring data disclosure.

To operationalize this, the system workflow is: 1) The experiment coordinator deploys a smart contract with a committed random seed (e.g., a VRF output). 2) For each user, the frontend client hashes a user identifier, computes their off-chain group assignment, and generates a ZK proof using the circuit. 3) The client submits only the user's public hash, the group index, and the proof to the coordinator's backend or directly to a verification contract. 4) The verifier checks the proof. If valid, the user is enrolled in the test, and their group is known to be correct without revealing why. This creates a cryptographically sound foundation for the subsequent private data collection and analysis phases.

The core privacy guarantee of our A/B testing framework is that raw user data never leaves the client device in a readable form. When a user interacts with a variant (e.g., clicks a new UI button), the client application must collect the relevant data and immediately encrypt it. This is done using a public key provided by the data aggregator. A common choice is the seal function from the NaCl cryptographic library (e.g., tweetnacl-js), which performs public-key authenticated encryption. The encrypted payload, or ciphertext, is the only data transmitted to the backend collector.

The data schema for encryption is critical. It must be structured to contain all necessary information for analysis while remaining compatible with the subsequent ZK-proof circuit. A typical payload in a JSON-like format before encryption might include: {"userId": "hashed_identifier", "variantId": "B", "event": "button_click", "timestamp": 1710451200}. The userId should be a deterministic hash (like SHA-256) of a stable identifier to enable user-level analysis without revealing the identifier itself. This structured plaintext is then encrypted, producing a binary ciphertext.

To enable the generation of a zero-knowledge proof later, the client must also create a cryptographic commitment to the plaintext data. This is a one-way hash of the data, such as commitment = SHA256(userId || variantId || event || timestamp). This commitment is sent alongside the ciphertext. The ZK-proof will later demonstrate that the prover knows a plaintext that matches both this public commitment and decrypts to valid data, without revealing the plaintext itself during the proof verification phase.

A practical implementation in a web environment uses a service worker or a dedicated SDK. The flow is: 1) Capture interaction event, 2) Construct data object, 3) Generate hash commitment, 4) Encrypt object using aggregator's public key, 5) Transmit the tuple (ciphertext, commitment) to the collection endpoint. This ensures that even the entity receiving the data cannot view its contents without the corresponding private decryption key, which is held separately and only used in the secure proving environment.

This client-side process shifts the trust model. The application server (running the A/B test) only sees opaque data blobs. Privacy is no longer dependent on the server's promise not to peek; it's enforced by cryptography. The next step, Step 3: Secure Aggregation with ZK-Proofs, will cover how the aggregator processes these encrypted blobs to compute statistics like conversion rates, using ZK-proofs to validate the integrity of the computation on the hidden data.

The circuit is the heart of the private A/B testing system. It defines the constraints that a valid proof must satisfy, encoding the statistical test (e.g., a two-proportion Z-test) as arithmetic operations over a finite field. We use Circom 2.0 for its developer-friendly syntax and integration with the Groth16 proving scheme. The primary inputs are the private witness data (user assignments and outcomes) and the public outputs (the aggregated test statistic and p-value). The circuit's job is to compute these outputs from the inputs while enforcing that all intermediate calculations are correct.

Our circuit must perform several key operations: 1) Securely aggregating counts by summing encrypted user outcomes per variant, 2) Calculating proportions (e.g., conversion rates) from these totals, 3) Computing the pooled standard error and the final Z-score, and 4) Deriving a p-value threshold check. Each step is translated into arithmetic constraints. For example, ensuring a sum is correct involves a component like Sum = a + b, which the prover must satisfy with valid witness values for a, b, and Sum.

A critical design pattern is the use of private signals and public signals. The raw user data—each user's variant assignment (0 for control, 1 for treatment) and converted flag (0 or 1)—are private signals. The circuit outputs, like the final Z-score and a boolean isSignificant flag (e.g., p-value < 0.05), are public signals. The prover supplies the private data, the circuit computes the public result, and the verifier only sees the public output and proof, never the individual user records.

Here is a simplified Circom template for the core aggregation and proportion logic:

circomCopy
template ABTestCircuit(n) {
    // Private Inputs
    signal input variant[n];
    signal input converted[n];
    
    // Public Outputs
    signal output controlConversions;
    signal output treatmentConversions;
    signal output controlRate;
    signal output treatmentRate;
    
    // Internal signals for counts
    signal controlCount;
    signal treatmentCount;
    
    // Calculate total conversions per group using a selector
    component controlSum = Sum(n);
    component treatmentSum = Sum(n);
    for (var i = 0; i < n; i++) {
        // If variant[i] == 0, add to control, else 0
        controlSum.in[i] <== converted[i] * (1 - variant[i]);
        // If variant[i] == 1, add to treatment, else 0
        treatmentSum.in[i] <== converted[i] * variant[i];
    }
    controlConversions <== controlSum.out;
    treatmentConversions <== treatmentSum.out;
    // ... Rate and Z-score calculation would follow
}

After defining the core arithmetic, the circuit must include range checks to ensure data integrity. For instance, we must constrain variant and converted signals to be binary (0 or 1). Without these checks, a malicious prover could input variant = 5, breaking the logic. We use Circom's IsZero and IsEqual components to enforce variant * (1 - variant) === 0. Similar checks apply to conversion flags. These constraints are essential for the circuit's soundness and the trustworthiness of the resulting proof.

Finally, the circuit is compiled into R1CS (Rank-1 Constraint System) format and a proving key/verification key pair is generated. This setup is a trusted ceremony for production, but for development, a random toxic waste parameter (tau) suffices. The compiled circuit defines the exact computational graph that the prover will execute on their private data to generate a proof, and the verifier will use to check that proof against the published aggregated result.

After aggregating the encrypted metric data off-chain, the next step is to generate a zero-knowledge proof (ZKP). This proof cryptographically attests that the final result (e.g., 'Variant B had a 12% higher conversion rate') was correctly computed from the valid, unaltered user data submitted in the previous steps. Using a ZK-SNARK circuit—for instance, written in Circom or Noir—you program the logic of your statistical test (like a chi-squared test or t-test). The proof demonstrates that all computations followed the protocol rules without exposing any individual user's data, variant assignment, or raw metric value.

The proof generation process requires a trusted setup to create proving and verification keys, a one-time ceremony for each circuit. For development, you can use tools like snarkjs with a Powers of Tau ceremony. The prover (typically your backend server) uses the proving key, the private witness (the aggregated encrypted data and the secret decryption key), and the public inputs (the test result) to generate a succinct proof. This proof is tiny, often just a few hundred bytes, making it cheap to transmit and verify. Popular libraries for this include arkworks in Rust or various JavaScript SDKs for different proving systems.

Once generated, the proof and the public result are sent to a verifier smart contract on-chain. This contract, pre-loaded with the verification key from the trusted setup, runs the verifyProof() function. The on-chain verification is computationally intensive for the prover but extremely cheap for the blockchain to check; it only requires a few elliptic curve operations. A successful verification transaction serves as a permanent, tamper-proof record that the announced A/B test outcome is valid. This allows anyone—team members, auditors, or the public—to trust the result without needing to trust the entity that ran the test.

This architecture decouples expensive computation from cheap verification. The heavy lifting of proof generation happens off-chain, while the immutable, trust-minimized verification is on-chain. You can use any EVM-compatible chain like Ethereum, Polygon, or Arbitrum for verification, with costs ranging from a few cents to a few dollars depending on gas prices. The contract's event logs can then trigger downstream actions, such as automatically releasing payments, updating protocol parameters, or publishing the verified result to a dashboard.

For developers, the key integration points are: 1) The circuit code defining the statistical test, 2) The off-chain prover service, and 3) The Solidity verifier contract. Frameworks like zkSync's ZK Stack or Polygon zkEVM provide tooling to compile circuits and generate verifier contracts automatically. This step transforms your private analytics framework from a conceptual design into a system with cryptographic guarantees, enabling transparent and compliant data analysis in Web3 applications.

The primary components of the framework are the prover (which generates a ZK-SNARK proof of correct metric computation) and the verifier (which checks the proof on-chain). By using a zk-friendly hash function like Poseidon and a proving system such as Groth16 or Plonk, you can create a circuit that accepts private user data and outputs only the aggregate result—like conversion rate or average session duration—alongside a validity proof. This ensures data confidentiality while maintaining auditability.

For production deployment, several critical steps remain. First, circuit optimization is essential to reduce proving time and cost. Techniques include minimizing non-linear constraints, using lookup tables for complex operations, and selecting optimal elliptic curve parameters (e.g., BN254 for Ethereum, BLS12-381 for broader compatibility). Second, you must implement secure and efficient off-chain data handling, potentially using trusted execution environments (TEEs) or secure multi-party computation (MPC) for the initial data aggregation before proof generation.

The next logical step is to explore advanced use cases. This framework can be extended to multi-armed bandit algorithms for dynamic traffic allocation, where the reward function for each variant is computed privately. Another direction is cross-chain A/B testing, where the verification of an experiment's outcome is settled on one chain but can be trustlessly referenced by applications on others via interoperability protocols like LayerZero or Axelar.

To continue your development, we recommend reviewing the following resources: the Circom documentation for circuit design, the SnarkJS library for proof generation, and audit reports from similar projects like Semaphore. Testing with frameworks such as Hardhat or Foundry, and using development networks like Anvil, will allow you to simulate gas costs and refine your contract logic before a mainnet deployment.

Finally, remember that the trust model shifts from trusting a central data processor to trusting the correctness of the cryptographic setup and the integrity of the smart contract. A robust implementation requires careful consideration of the trusted setup ceremony for your proving system, continuous monitoring for circuit bugs, and clear documentation for experiment stakeholders on how to interpret the verifiable results published on-chain.