ProvenanceGuard Adds Source-Aware Verification to MCP Agents
ProvenanceGuard checks whether MCP agent claims cite the right source, catching 138 of 139 blocked claims but overblocking 67 supported claims in tests.
Summary
On September 29, 2026, Multiverse Computing detailed ProvenanceGuard, a post-generation verifier that detects cross-source conflation in Model Context Protocol agents. Without retraining a black-box agent, it preserves source IDs from captured tool traces, decomposes answers into claims, routes each claim to relevant evidence, verifies support and attribution, then allows, blocks or repairs the answer. The evaluated offline configuration used MiniLM, a DeBERTa NLI verifier and a local language model, with strict checks for numbers, dates and identifiers.
Across 281 medical-agent traces, experts evaluated 361 claims from 40 held-out answers. ProvenanceGuard caught 138 of 139 claims marked for rejection, held 67 supported claims for review or repair, and selected the correct identifiable source about 86% of the time. Its reject F1 was 0.802, versus MiniCheck at 0.783, RAGAS Faithfulness at 0.758, AlignScore at 0.662 and SummaC-ZS at 0.436. It caught all 50 deliberately swapped attributions; with several similar sources, block F1 reached 0.846 but exact-source accuracy fell to 50.3%.
RARR-style repair resolved all 173 blocked full-trace answers, although 144 became fallback text, and all 59 blocked reconstructed multi-source cases, with two terminal fallbacks. Reported offline latency was roughly half a second per answer. NVIDIA NVFlow has added optional ProvenanceGuard-based verification against retrieved SEC excerpts for its finance agent, while the research was presented at UC Berkeley's Agentic AI Summit 2026.
Positives
- 138 of 139 expert-rejected claims were caught, leaving one unsupported claim undetected.
- 0.802 reject F1 exceeded MiniCheck, RAGAS Faithfulness, AlignScore and SummaC-ZS on the held-out claims.
- All 50 deliberately altered source attributions were detected while the supporting evidence remained unchanged.
- All 173 full-trace and 59 reconstructed multi-source blocked answers were resolved through repair or fallback.
- NVIDIA NVFlow added optional source-aware checks against SEC excerpts without changing its finance agent's rollout or training data.
Risks & concerns
- 67 expert-supported claims were held for review or repair under the conservative decision policy.
- 50.3% exact-source accuracy in the similar-source test shows routing remains unreliable when evidence sources closely resemble one another.
- 144 of 173 blocked full-trace answers ended as fallback text rather than substantive rewrites.
- One of 139 claims experts said should be rejected passed verification.
- New hosted-model configurations require separate testing and calibration because reported results cover the local setup.