publications

2022

1. VOTING 22

   Simulations of Ballot Polling Risk-Limiting Audits

   Oliver Broadrick, Sarah Morin, Grant McClearn, Neal McBurnett, Poorvi L. Vora, and Filip Zagórski

   In Seventh Workshop on Advances in Secure Electronic Voting, Financial Cryptography, 2022

   Abs PDF

   In this paper we present simulation results comparing the risk, stopping probability, and number of ballots required over multiple rounds of ballot polling risk-limiting audits (RLAs) Minerva, Selection-Ordered (SO) Bravo, and End-of-Round (EoR) Bravo . Bravo is the most commonly used ballot polling RLA and requires the smallest expected number of ballots when ballots are drawn one at a time and the (true) underlying election is as announced. In real audits, multiple ballots are drawn at a time, and Bravo is implemented as SO Bravo or EoR Bravo.

   Minerva is a recently proposed ballot polling RLA that requires fewer ballots than either implementation of Bravo in a first round with stopping probability 0.9 but requires a predetermined round schedule. It is an open question how these audits compare over multiple rounds and for lower stopping probabilities. Our simulations use stopping probabilities of 0.9 and 0.25. The results are consistent with predictions of the R2B2 open-source library for ballot polling audits. We observe that both Bravo audits are more conservative than Minerva, which stops with fewer ballots, for both first round stopping probabilities. However, the advantage of using Minerva decreases considerably for the smaller first round stopping probability, as one would expect.

2021

1. USENIX Security

   Minerva– An Efficient Risk-Limiting Ballot Polling Audit

   Filip Zagórski, Grant McClearn, Sarah Morin, Neal McBurnett, and Poorvi L. Vora

   In 30th USENIX Security Symposium (USENIX Security ’21), Aug 2021

   Abs PDF

   Evidence-based elections aim to produce trustworthy and compelling evidence of the correctness of election outcomes, enabling the detection of problems with high probability. They require a well-curated voter-verified paper trail, compliance audits, and a rigorous tabulation audit of the election outcome, known as a risk-limiting audit (RLA).

   This paper focuses on ballot polling RLAs which can require that a very large sample of ballots be drawn. The main ballot polling RLA in use today, BRAVO, is designed for use when single ballots are drawn at random and a decision regarding whether to stop the audit or draw another ballot is taken after each ballot draw. But in practice, ballot polling audits draw many ballots in a single round before determining whether to stop.

   Direct application of BRAVO to large rounds results in considerable inefficiency. We present MINERVA, a risk-limiting audit that addresses this problem. When compared to the BRAVO stopping rule being applied at the end of the round, for a first-round with 90% stopping probability, MINERVA halves the number of ballots required across all state margins in the 2020 US Presidential election. When compared to the BRAVO stopping rule being applied after examination of individual ballots, MINERVA reduces the number of ballots by about a quarter. MINERVA requires that round sizes are predetermined; this does not appear to be a drawback for large first rounds which have been typical choices for election officials.

   Ballot-polling audits are the leading option in most states. MINERVA significantly reduces the necessary expense for contests with close margins and thus makes adopting RLAs easier. Wider adoption of RLAs is a critical step in increasing public confidence in elections.

   MINERVA was used in Ohio’s pilot RLA of the primaries in May 2020 in Montgomery County. We provide open-source implementations of MINERVA. The code has been integrated as an option in Arlo, the most widely-used RLA software.

2. The Athena Class of Risk-Limiting Ballot Polling Audits

   Filip Zagórski, Grant McClearn, Sarah Morin, Neal McBurnett, and Poorvi L. Vora

   CoRR, 2021

   Abs arXiv PDF

   The main risk-limiting ballot polling audit in use today, BRAVO, is designed for use when single ballots are drawn at random and a decision regarding whether to stop the audit or draw another ballot is taken after each ballot draw (ballot-by-ballot (B2) audits). On the other hand, real ballot polling audits draw many ballots in a single round before determining whether to stop (round-by-round (R2) audits). We show that BRAVO results in significant inefficiency when directly applied to real R2 audits. We present the ATHENA class of R2 stopping rules, which we show are risk-limiting if the round schedule is pre-determined (before the audit begins). We prove that each rule is at least as efficient as the corresponding BRAVO stopping rule applied at the end of the round. We have open-source software libraries implementing most of our results.

   We show that ATHENA halves the number of ballots required, for all state margins in the 2016 US Presidential election and a first round with 90% stopping probability, when compared to BRAVO (stopping rule applied at the end of the round). We present simulation results supporting the 90% stopping probability claims and our claims for the risk accrued in the first round. Further, ATHENA reduces the number of ballots by more than a quarter for low margins, when compared to the BRAVO stopping rule applied on ballots in selection order. This implies that keeping track of the order when drawing ballots R2 is not beneficial, because ATHENA is more efficient even without information on selection order. These results are significant because current approaches to real ballot polling election audits use the B2 BRAVO rules, requiring about twice as much work on the part of election officials. Applying the rules in selection order requires fewer ballots, but keeping track of the order, and entering it into audit software, adds to the effort.

2020

1. VOTING 20

   A Note on Risk-Limiting Bayesian Polling Audits for Two-Candidate Elections

   Sarah Morin, Grant McClearn, Neal McBurnett, Poorvi L. Vora, and Filip Zagórski

   In Fifth Workshop on Advances in Secure Electronic Voting, Financial Cryptography, 2020

   Abs PDF

   This short paper provides a general form for a polling audit that is both Bayesian and risk-limiting: the Bayesian Risk-Limiting (Polling) Audit, which enables the use of a Bayesian approach to explore more efficient Risk-Limiting Audits. A numerical example illustrates the implications to practice.