OpenAI Evals
OpenAI's original open-source eval framework and registry; largely superseded by the hosted Evals API but still a reference implementation.
| Category | Evals & Testing |
| Open source | Yes |
| Self-hostable | Yes |
| Pricing model | free |
| Pricing notes | MIT OSS framework; OpenAI platform also has a hosted Evals product billed via API usage |
| Framework integrations | openai-sdk |
| Funding / ownership | OpenAI |
| GitHub stars | 18,683 |
| Maintenance | maintained (commit 2026-04-14) |
| License (GitHub) | not detected |
| Open issues | 208 |
| Maturity signal | 80/100 (Mature) — computed from public GitHub signals, see methodology |
Pricing/feature source: https://github.com/openai/evals
Maturity signal is computed from public GitHub data only. How it is calculated.
Frequently asked questions
Is OpenAI Evals open source?
Yes.
Can I self-host OpenAI Evals?
Yes — OpenAI Evals can run in your own infrastructure.
How much does OpenAI Evals cost?
Free. MIT OSS framework; OpenAI platform also has a hosted Evals product billed via API usage