Testing an Endpoint
The Test an Endpoint page, opened with + Test Endpoint, sets up one test. This guide covers every field, option and panel on it.
The Fields
| Field | What to enter |
|---|---|
| Endpoint Base URL | The endpoint's address, usually ending in /v1. It must be reachable from the internet: addresses on a private or local network, such as localhost, are refused. |
| Model ID at the Endpoint | The model name the endpoint expects in requests. List fetches the endpoint's model list and opens it under the field: type to narrow it down, then pick with a click, or with the arrow keys and Enter. A name the endpoint doesn't list can still be typed. |
| Claims to Be | The model the endpoint says it serves. The test compares the answers with that model's official answers. |
| API Key | Your key for the endpoint. It's used for this test only and never stored. If the key's balance runs out, the test fails. |
| Baseline | Shown when more than one set of reference data is installed. The newest is selected; to run the competence check, pick one that offers it for the claimed model. Baselines lists which models support which checks. |
Model ID at the Endpoint and Claims to Be are often different names for the same model. OpenRouter, for example, calls GLM 5.3 z-ai/glm-5.3.
API Formats
| API | When to use it |
|---|---|
| Chat Completions | The OpenAI-compatible format. Most relays and resellers use it, and it's the default. |
| Responses | Not available yet. OpenAI's newer Responses API stays off until tests through it are checked against OpenAI's own service. |
| Claude Messages | Not available yet. Anthropic's Messages API, used by relays that resell Claude in that format, stays off until tests through it are checked against Anthropic's own service. |
Request settings that an API can't express are refused, not silently dropped. Where a model refuses a setting it would use anyway, such as a temperature of 1 on current Claude models, it's left out.
Tiers
| Tier | Identity probes | What it's for |
|---|---|---|
| Standard | 80 | A first look. It tells the baseline's models apart in nearly every case. |
| Thorough | 160 | Close relatives, small shares of routed traffic, and confirming any verdict before you act on it. |
The refusal and competence checks double with the tier as well.
Options
- Competence Check (off by default) sends test items with exactly computable answers, to catch a model served below official quality. It's available only where the baseline holds official competence data for the claimed model; otherwise the switch is greyed out with "no plan for this model". See Competence Check.
- Refusal Check (off by default) sends benign requests that the official model declines, to catch a copy with its refusals removed. See Refusal Check.
- Stream (on by default) receives answers as a stream. Keep it on: some relays cut off requests that run longer than about a minute, and reasoning models often do.
The Side Panel
- This Test Sends counts the requests per kind, and their total.
- Limit Field, Max Tokens and Temperature show what each request carries. These are fixed by the baseline: a test sends exactly what the official answers were collected with, so the comparison is fair.
- What This Tier Can Tell gives, for the claimed model at this tier, how often a genuine endpoint is confirmed, how often it's wrongly flagged, how reliably substitutes are caught, and the catch rates of the competence and refusal checks. These numbers come from the baseline's own simulations.
- Possible Verdicts shows the verdict colors used across the app.
While a Test Runs
The report page fills in as answers arrive: the progress bar, the closest model so far, and errors. Stop ends the test early. A stopped test is judged on the answers it has, and with too few it ends Inconclusive.
Rate Limits
A request refused by a rate limit is sent again until three minutes after its first refusal. These resends don't count as attempts. Every test of the same model at the same endpoint with the same key waits out a cooldown together. Once 4 requests in a row are still refused, the test stops sending and ends Inconclusive with a note saying why. Run it again later.
OpenRouter Keys
For every request in flight, OpenRouter reserves the cost of its prompt plus the maximum output, and refuses requests that don't fit (HTTP 402). Small or new accounts can only fund a few requests at a time. Those refusals are sent again as other requests finish, so a test on a small account runs slower but still completes. If the credit runs out, the report says so.
Running a Test Again
Rerun on a report fills in this form with that test's endpoint, model, claim, API and tier. Enter the key again; it's never kept.