Data Analysis for PolicyMaking in Georgia
Module 5 · Building Your Policy Finding › 5.4 Sanity-checking your numbers
Subsection 5.4

Sanity-checking your numbers

~3 min

Reading

Before you publish any finding, sanity-check your numbers. Data analysis errors are common. Spreadsheet formula errors, unit mismatches, and incorrect joins are responsible for a significant share of incorrect findings in policy research. A brief sanity check process can catch most errors before they become embarrassing—or harmful.

Gut checks

The first step is simply asking: does this number make sense? If your calculation shows that 110% of registered voters voted, something is wrong. If it shows that a county had zero mail ballot rejections in 2020 but 2,000 in 2022, you need to check whether the data structure changed (was rejection data recorded differently in those years?). Numbers that are too round, too extreme, or inconsistent with known facts are all signals to investigate.

Magnitude checks

Compare your number to similar, known quantities. If you're calculating Georgia's 2020 statewide turnout and you get 42%, check that against certified totals from the Secretary of State—Georgia's 2020 turnout was actually around 66% of active registered voters. If your number is off by more than a few percentage points from a known benchmark, trace the difference before proceeding.

Checking against known benchmarks

Key Georgia benchmarks to know and check against:

  • 2020 general election statewide turnout: ~66% of active registered voters
  • 2022 general election statewide turnout: ~54% of active registered voters (midterm)
  • 2024 general election statewide turnout: approximately 57–58% of active registered voters
  • Active registered voters (2024): approximately 8.0–8.1 million
  • 159 counties in Georgia
  • 2020 presidential election in Georgia: Biden won by 11,779 votes (0.2% margin)

Documenting your sanity check

Your methodology note should include a brief statement that your numbers were checked against official certified results and/or published benchmarks. This is a practice that distinguishes professional-grade policy analysis from casual data work. When a reporter or opposing counsel asks "how do you know your numbers are right?", your sanity check documentation is your answer.

Common error sources in Georgia voter data

Watch for these specifically: (1) including inactive voters in your denominator when calculating turnout; (2) using county FIPS code instead of GA county code in a join; (3) double-counting split precincts that appear in multiple district assignments; (4) using the wrong election-year column in voting history flags.

Sanity Check Checklist — Before Publishing Any Finding What to check ☐ Gut check: is this number in the plausible range? ☐ Compare to certified/official source (SoS, Census) ☐ Denominator: active registrants only (not inactive)? ☐ Join key: GA county code vs. FIPS — are you using the right one? ☐ Split precincts: check for double-counting ☐ Voting history column: right election year flag used? ☐ Rates vs. counts: consistent across all compared periods ☐ Document: what you excluded and how many rows Georgia Benchmarks Active registered voters 2024: ~8.0–8.1 million 2024 general turnout: ~57–58% of active reg. 2022 general turnout: ~54% of active reg. 2020 general turnout: ~66% of active reg. 2020 presidential margin: Biden +0.2% (11,779 votes) Number of GA counties: 159 Source: Georgia Secretary of State certified results
Diagram 5.4 · Sanity check checklist. Run these checks before publishing any finding. Left: checks to perform. Right: Georgia benchmarks to compare against.