Theoria: Rewrite-Acceptability Verification over Informal Reasoning States
Ben Slivinski, Michael Saldivar
Read on arXiv →
Key claim
Theoria certifies AI outputs with high precision and transparency.
In plain English
Imagine you're using an AI to solve complex problems, like legal questions or technical challenges. You want to trust its answers, but how do you know if it's right? Current methods either give you certainty but miss many problems, or they provide broad coverage but lack transparency, making it hard to audit their decisions. This is where Theoria comes in. It rewrites potential solutions into a series of clear, justified steps, allowing each transition to be independently verified. This means if something goes wrong, you can trace it back to where the AI made a mistake. In tests, Theoria certified over 91% of expert problems with a clear proof for each step, while traditional methods struggled with hidden errors. This approach not only improves accuracy but also builds trust, as users can see and challenge each part of the reasoning process. For anyone building AI systems that require reliable outputs, Theoria offers a practical way to ensure that the answers you get are not just confident but also correct.
Theoria introduces a new verification architecture that enhances trust in AI outputs.
The results are supported by rigorous testing on multiple datasets with clear metrics.
Deep reliability assessment
The methodology supports the claim that Theoria can provide structured verification with high precision, but the reliance on LLM judges without formal backstopping may overclaim robustness against adversarial inputs.
Reproducibility
Yes, the paper mentions an open-source code repository: github.com/zaladbar/theoria.
Key figure
The key architectural diagram illustrates Theoria's process of rewriting a candidate solution into a sequence of typed state transitions, each with explicit justification, allowing for independent auditing of each transition.