An ETHLabs research track on one question: when an AI answers a question about an Indian company's filings, can each statement in that answer be traced back to a page that supports it?
Language models can state figures and facts about a company that its own filings do not support, and they state them in the same confident tone as the correct ones.
The study asks whether adding a claim-level check to a retrieval-based pipeline changes how often unsupported statements appear on Indian filings, and which kinds of errors remain.
In finance, an unsupported figure can travel into a decision. A trustworthy system has to be able to show where each statement came from, or decline to make it.
Schematic only. The content of the claims, the checker and the sources are withheld until the work is published.
The design is held back on purpose. It is released with the paper, not before.
Pipelines compared:
Checking stage:
Models used:
Companies studied:
Question set:
Reference answers:
Judging protocol:
Error taxonomy:
Statistics:
Source code:
Annotated set:
Release:
Research question and scope defined.
Systems and test material are being prepared.
Answers are judged by a person against the filings.
Findings, including the failures that remain, are written up and released.
No findings are reported here. Numbers appear on this page only after they have been measured and written up.
Every statement traces to a source page.
No result is stated before it is measured.
Failures are reported alongside successes.
Dhiraj Raj Singh
Researcher, ETHLabs