Agenta
Open-source LLMOps platform for prompt management, evaluation, and deployment
Visit Product
83 upvotes
2,574 views
About
Agenta is an open-source LLMOps platform that provides everything a team needs to build, test, and deploy reliable LLM applications. It creates a collaborative environment where both engineers and non-technical stakeholders can participate in prompt engineering, evaluation, and quality assurance — bridging the gap between AI development and product teams.
The platform's prompt playground supports complex multi-step prompts, chains, and agent workflows. Teams can create and manage prompt variants, run systematic evaluations against test datasets, and track how different prompt versions perform over time. Agenta's annotation tools enable human-in-the-loop evaluation workflows where domain experts can review and rate AI outputs.
As an open-source project, Agenta can be self-hosted within any infrastructure, making it suitable for companies with strict data governance requirements. The platform supports integration with all major LLM providers and custom model endpoints, giving teams flexibility in their model choices.
The platform's prompt playground supports complex multi-step prompts, chains, and agent workflows. Teams can create and manage prompt variants, run systematic evaluations against test datasets, and track how different prompt versions perform over time. Agenta's annotation tools enable human-in-the-loop evaluation workflows where domain experts can review and rate AI outputs.
As an open-source project, Agenta can be self-hosted within any infrastructure, making it suitable for companies with strict data governance requirements. The platform supports integration with all major LLM providers and custom model endpoints, giving teams flexibility in their model choices.
Product Features
- Collaborative prompt playground with variant management
- Evaluation suite with automated and human review modes
- Test dataset management and versioning
- Deployment to REST API endpoints from the UI
- LLM provider agnostic: OpenAI, Anthropic, Cohere, custom
- Annotation and human feedback collection
- Tracing for debugging complex LLM applications
- Docker-based self-hosting
- Python SDK for programmatic access
- Role-based access control for teams
- Evaluation suite with automated and human review modes
- Test dataset management and versioning
- Deployment to REST API endpoints from the UI
- LLM provider agnostic: OpenAI, Anthropic, Cohere, custom
- Annotation and human feedback collection
- Tracing for debugging complex LLM applications
- Docker-based self-hosting
- Python SDK for programmatic access
- Role-based access control for teams
About the Publisher
Agenta was founded by Abram Zogott and Mahmoud Mabrouk with a mission to make LLM engineering more systematic and collaborative. The open-source community around Agenta has grown rapidly, with contributions from developers across the world who share the vision of making LLM development as rigorous as traditional software engineering. The platform is backed by Y Combinator and serves development teams at technology companies building production AI products.