TensorZero
Open-source framework for building and optimizing production LLM applications
Visit Product
275 upvotes
456 views
About
TensorZero is an open-source framework that treats LLM application development as a systematic optimization problem rather than an art form. It provides a structured way to define, deploy, and continuously improve LLM functions — capturing feedback signals from production and using them to automatically improve prompts and model configurations through machine learning.
At its core, TensorZero introduces the concept of "LLM functions" — typed, versioned units of LLM capability that can be A/B tested, evaluated, and optimized based on real usage data. Rather than manually tweaking prompts and hoping for improvement, TensorZero enables data-driven optimization that finds better configurations through systematic experimentation and feedback-based learning.
The framework integrates with all major LLM providers and includes built-in support for common patterns like structured output, tool use, and multi-step reasoning. Its Rust-based inference gateway provides high performance and low latency suitable for production traffic at scale.
At its core, TensorZero introduces the concept of "LLM functions" — typed, versioned units of LLM capability that can be A/B tested, evaluated, and optimized based on real usage data. Rather than manually tweaking prompts and hoping for improvement, TensorZero enables data-driven optimization that finds better configurations through systematic experimentation and feedback-based learning.
The framework integrates with all major LLM providers and includes built-in support for common patterns like structured output, tool use, and multi-step reasoning. Its Rust-based inference gateway provides high performance and low latency suitable for production traffic at scale.
Product Features
- LLM function abstraction with typed inputs and outputs
- A/B testing of prompts and model configurations
- Feedback collection from production usage
- Automatic prompt and model optimization
- High-performance Rust inference gateway
- Structured output enforcement with JSON schema
- Multi-provider support with fallback routing
- Full observability with request logging
- Integration with OpenAI, Anthropic, and other providers
- Python and TypeScript client libraries
- A/B testing of prompts and model configurations
- Feedback collection from production usage
- Automatic prompt and model optimization
- High-performance Rust inference gateway
- Structured output enforcement with JSON schema
- Multi-provider support with fallback routing
- Full observability with request logging
- Integration with OpenAI, Anthropic, and other providers
- Python and TypeScript client libraries
About the Publisher
TensorZero was founded with the conviction that LLM applications should be optimized using the same rigorous, data-driven methods applied to traditional ML systems. The team brings together expertise in systems engineering, machine learning, and product development. TensorZero is open-source and community-driven, welcoming contributions from developers who share the vision of making LLM application development more scientific and reliable.