Proof-Carrying Numbers: Ensuring Trust in AI-Driven Development Data

The Problem: AI and Data Integrity

As AI becomes more prevalent in accessing and interpreting development data, a critical issue arises: how do we guarantee the accuracy of the numbers these systems produce? Large language models (LLMs) can quickly summarize reports and answer data queries, but their outputs may subtly deviate from the original data, potentially misleading policymakers and the public.

AI Access Doesn't Equal Data Integrity

Many AI systems use retrieval-augmented generation (RAG), feeding models with trusted sources. However, this doesn't eliminate inaccuracies. Even when AI retrieves correct data, it might round numbers or merge values, altering their meaning. These changes often go unnoticed because the AI appears confident, creating a false sense of security. Current evaluation methods, which report aggregate accuracy rates, don't help those who need to rely on a specific number. The focus should be on whether each individual data point is faithful to the original source.

Proof-Carrying Numbers (PCN): A New Approach

Proof-Carrying Numbers (PCN) offer a novel solution by verifying the faithfulness of AI-generated numbers in real-time. This protocol, developed by the AI for Data - Data for AI team, introduces a mechanism to check how closely an AI's numbers match the original data.

How PCN Works

The process involves several steps. First, the data provided to the LLM includes a claim identifier and a policy defining acceptable behavior, such as whether rounding is allowed. Next, the model follows the PCN protocol when generating numbers. Each output is then checked against the original data. If the result aligns with the policy, PCN marks it as verified with a checkmark [✓]. If it deviates, it's flagged for review [⚠️]. Numbers without these marks should be treated with caution. This approach acts as a fail-closed mechanism, prioritizing caution and making any failures visible. It shifts user interaction from blind trust to informed assessment.

PCN vs. Citation-Based Solutions

Many AI systems rely on citations to build trust. However, citations alone don't guarantee accuracy. Models can still misstate or reinterpret numbers even while citing legitimate sources, creating a false sense of credibility. PCN goes further by performing numerical verification. It checks every value against the underlying data and produces a verification mark accordingly. Citations show where information comes from, while PCN enforces faithfulness.

Why This Matters for Official Statistics

Official statistics underpin development policy and are curated with rigorous standards. However, AI's probabilistic nature can undermine the trust in these numbers. PCN helps bridge this gap by embedding governance directly into AI-mediated dissemination, ensuring that no number appears official unless it matches the authoritative dataset. It also improves transparency by clearly signaling when deviations occur and strengthens accountability by making verification explicit.

From Trust by Design to Governance by Proof

Traditional systems rely on metadata to help users interpret statistics. While metadata describes what a number means, it cannot enforce correctness. PCN introduces verification instead of description. It complements metadata by automatically checking whether an AI system's outputs remain faithful. This marks a shift from trusting systems by design to verifying their behavior every time a number is generated.

A Foundation for Responsible AI

As institutions explore AI for data dissemination, PCN offers a way to strengthen governance and make data systems AI-accountable. Future interfaces can display numbers that are aligned with official data or clearly marked when they are not, keeping users informed and institutions authoritative. The result is a model that preserves the credibility of official statistics while expanding their reach.

Looking Ahead

AI-assisted access to development data is inevitable, promising to democratize information. However, innovation without safeguards risks undermining trust. PCN offers this safeguard by coupling AI generation with real-time verification. It ensures that every number carries not just context but proof, maintaining governance in the age of AI. PCN is not just technical; it's a governance innovation that helps preserve the integrity of official statistics as AI becomes a primary channel for data access.