Anthropic develops open source system to verify which AI model produced an output
The system, called FlashProve, would give users a cryptographic proof that an answer came from a specific model, TechCrunch reported. It is still a research project.
Anthropic is working on an open source system meant to verify which AI model produced a given output, according to TechCrunch, which reported on a blog post the company published Tuesday.
The system is called FlashProve. TechCrunch reported that it is designed to provide what researchers call "provable inference": a cryptographic proof that a given output came from a specific model. The goal is to let users confirm that an answer really came from the model they think they are using.
TechCrunch compared it to cryptographic email signatures, which let recipients confirm that a message came from its claimed sender. According to the report, FlashProve is designed to resist impostors trying to pass one model off as another, even when the impostor is the lab that built and runs the model.
That is where the researchers see much of its value. According to TechCrunch, they note that the system could stop an AI company from secretly swapping in a cheaper model to cut costs, or a more expensive one to inflate its performance results.
It could also be used to check claims by third parties that an advanced model did a particular task. The researchers gave examples in the post, as quoted by TechCrunch:
“"A developer might claim that their software library was scanned for vulnerabilities by a frontier cybersecurity model, or a company might claim that its business plan was given the go-ahead by an appropriate model. How can a user verify these claims?"”
For the system to work, AI labs would need to publish a "fingerprint" of each model they release. That fingerprint could later be matched to specific outputs and checked cryptographically. TechCrunch reported that the fingerprints do not reveal a model's weights, the internal numbers that define how it behaves. Closed-model makers such as Anthropic and OpenAI keep those private, so labs could publish fingerprints without giving away their most valuable asset. That would let the system work with commercial models too. It is also designed to be lightweight, so verification could run on a local device.
Alongside the announcement, the researchers released a detailed technical specification and working code for parts of the system on GitHub under an open source license, TechCrunch reported.
Much remains unsettled. According to TechCrunch, research is ongoing and there are no plans to deploy the system at Anthropic or any other lab. The report does not say whether other labs have agreed to publish model fingerprints, or when, if ever, the system might be ready for use.
The researchers described the work as unfinished.
“"This research post is meant to share intermediate progress, and not a completed research agenda. We are sharing this explicitly with the goal that others look at what we have done, take it, and improve it."”Anthropic researchers, in the announcement post