Is your API relay watering down your models? A detection methodology
2026-06-17 · LLM, API relay, model detection, fingerprinting
LLM API relays (reselling Claude/GPT/Gemini access) have an open secret: upstream channels often "water down" — the dashboard says Claude Sonnet, but requests are quietly routed to far cheaper models (Qwen, DeepSeek, or quantized degraded versions). Worse, the swapped upstream is often prompted to claim it is still Claude, so asking it directly is useless.
Why asking the model is useless
In our controlled test, injecting a "you are Claude" system prompt made cheap models claim to be Claude 100% of the time. Identity must come from behavioral fingerprints — the model's output distribution under controlled inputs is its handwriting, which a disguise prompt cannot change.
How behavioral-fingerprint detection works
A curated set of probe inputs is sent to the endpoint; outputs are turned into a vector fingerprint and compared against reference fingerprints of major model families. If the claimed family differs from the fingerprinted family, it is suspected substitution. The approach has solid academic grounding (e.g. LLMmap, USENIX Security 2025). Even when the upstream is prompted to claim it is Claude, the fingerprint still points to its real family.
Accuracy and limits (honest)
Whole-model substitution (Claude to Qwen) is detected with high accuracy (~96%, consistent with published work). Under disguise, self-report is fooled but the fingerprint still identifies the real family. Weak-fingerprint families can false-positive on a single run; multi-round voting fixes this. Quantized/distilled degraded versions of the same model are hard to tell apart; models outside the reference set can only be flagged as "not any known one". All results are probabilistic signals, not legal proof.
Test it yourself
We built a free checker: panshi.io/relay-check — paste model output on the web, or run a Python script locally with your own relay key (the key never leaves your machine). Relay operators wanting continuous monitoring or a relay system with built-in QA: see Relay Guard.