Confident AI (DeepEval)
Pytest-style open-source LLM evaluation framework (DeepEval) with a hosted platform for benchmarking, regression testing and red-teaming (DeepTeam).
| Category | Evals & Testing |
| Open source | Yes |
| Self-hostable | Yes |
| Pricing model | freemium |
| Pricing notes | DeepEval framework free OSS (Apache-2.0); Confident AI cloud has free + paid tiers |
| Framework integrations | openai-sdk, langchain, llamaindex, anthropic |
| Funding / ownership | YC W25; $2.2M seed |
| GitHub stars | 16,132 |
| Maintenance | active (commit 2026-06-10) |
| License (GitHub) | Apache-2.0 |
| Open issues | 288 |
| Maturity signal | 100/100 (Mature) — computed from public GitHub signals, see methodology |
Pricing/feature source: https://www.confident-ai.com/pricing
Maturity signal is computed from public GitHub data only. How it is calculated.
Frequently asked questions
Is Confident AI (DeepEval) open source?
Yes. Licensed under Apache-2.0.
Can I self-host Confident AI (DeepEval)?
Yes — Confident AI (DeepEval) can run in your own infrastructure.
How much does Confident AI (DeepEval) cost?
Freemium. DeepEval framework free OSS (Apache-2.0); Confident AI cloud has free + paid tiers