# Limits

Read these before acting on a verdict.

- **A mismatch is evidence, not proof of intent.** Answer distributions also change with a system prompt added by a relay, different sampling defaults, quantization, a new checkpoint from the vendor, or routing. Confirm any mismatch with a separate test at the Thorough tier.
- **Baselines age.** Vendors update hosted models. If the vendor's own API suddenly comes out as Unknown Model, the baseline is out of date, not the vendor.
- **Models outside the baseline can't be named.** They're caught when their answers don't fit the claimed model. A close lookalike can pass a Standard test.
- **Simulated rates are averages.** The numbers in the side panel and on the Baselines page come from simulations on the collected data. For the closest pairs, a real test can do somewhat worse.
- **Token counts and model names in replies are hints.** Relays can rewrite both. They become evidence only when requests with different counts also answer differently.
- **Siblings can't be separated.** Models that answer almost alike, such as MiMo-V2.5 and MiMo-V2.6-Flash, end Inconclusive rather than being called substitutes.
- **Mixed routing needs a large share or different families.** A quarter of the traffic sent to a close relative is mostly missed. See [Mixed Routing](https://llm-id.sh/docs/mixed-routing).
- **The competence check is experimental.** It catches about half of clearly degraded deployments at Standard, and "consistent" doesn't prove full quality. See [Competence Check](https://llm-id.sh/docs/competence).
- **The refusal check is for complete removals.** A copy that complies half the time gets a note, not a verdict, and telling modified weights from a jailbreak prompt rests on token counts. See [Refusal Check](https://llm-id.sh/docs/refusals).

---

From the llm-id.sh guides: https://llm-id.sh/docs/limits