$ cat models/gpt-6.1-sol
GPT-6.1 Sol
OpenAI
Check that an endpoint selling GPT-6.1 Sol really serves it. A test compares the endpoint's answers with answers collected from OpenAI, through OpenRouter pinned to OpenAI.
- Official Provider
- OpenAI, through OpenRouter pinned to OpenAI
- Reference Answers
- 1,000
- Declared Precision
- closed model
- Checks
- Identity
- Reference Data Updated
- 2026-10-16
How Well a Test Works
Size
Requests
Genuine Confirmed
Wrongly Flagged
Substitutes Caught
Standard
80
57.7%
0.5%
32.8% · hardest GPT-6 Astra
Thorough
160
93.2%
0.5%
86.4% · hardest GPT-6 Astra
Simulated from the reference answers. A substitute that isn't caught ends inconclusive: it is never confirmed as this model.
- GPT-6.1 Sol and GPT-6 Astra answer almost alike: at Standard a genuine endpoint is confirmed in only 57.7% of tests, at Thorough in 93.2%. The rest end inconclusive; neither model is ever confirmed in the other's place. Test it at Thorough.
- No refusal check: GPT-6.1 Sol's weights aren't published, so nobody can abliterate it.
- No competence check yet: the competence check needs hard tasks collected from the official provider, which this model doesn't have.