Stable Beluga
by Stability AI
Fine-tuned LLaMA 65B model optimized for instruction following and reasoning
Visit Product
64 upvotes
764 views
About
Stable Beluga is a fine-tuned language model built on Meta's LLaMA 65B foundation, developed by Stability AI's CarperAI lab using Orca-style synthetic data training. The model was fine-tuned on a carefully curated dataset of reasoning examples that include step-by-step explanations — teaching the model to approach problems more thoughtfully and produce more reliable answers.
The Orca-style training methodology, pioneered by Microsoft Research, generates training data where large teacher models (GPT-4) provide detailed reasoning chains alongside answers. When a smaller model is trained on this data, it learns not just what the answer is but how to reason through the problem — resulting in significant quality improvements over standard instruction-fine-tuning approaches.
Stable Beluga demonstrated strong performance on academic reasoning benchmarks and was widely recognized as one of the most capable open-source instruction-following models at its release. It demonstrated that the same efficient training techniques developed for Stable Beluga 2 could significantly improve models at the 65B scale.
The Orca-style training methodology, pioneered by Microsoft Research, generates training data where large teacher models (GPT-4) provide detailed reasoning chains alongside answers. When a smaller model is trained on this data, it learns not just what the answer is but how to reason through the problem — resulting in significant quality improvements over standard instruction-fine-tuning approaches.
Stable Beluga demonstrated strong performance on academic reasoning benchmarks and was widely recognized as one of the most capable open-source instruction-following models at its release. It demonstrated that the same efficient training techniques developed for Stable Beluga 2 could significantly improve models at the 65B scale.
Product Features
- Fine-tuned on Orca-style reasoning data
- Built on LLaMA 65B for strong baseline capability
- Step-by-step reasoning approach
- Open weights available on Hugging Face
- Strong benchmark performance on MMLU and others
- Commercial use within LLaMA license terms
- Compatible with standard LLaMA inference frameworks
- Quantized versions available
- Research-grade model for ablation studies
- Predecessor to the improved Stable Beluga 2
- Built on LLaMA 65B for strong baseline capability
- Step-by-step reasoning approach
- Open weights available on Hugging Face
- Strong benchmark performance on MMLU and others
- Commercial use within LLaMA license terms
- Compatible with standard LLaMA inference frameworks
- Quantized versions available
- Research-grade model for ablation studies
- Predecessor to the improved Stable Beluga 2
About the Publisher
Stability AI founded CarperAI, a research division focused on open-source language model research, which produced the Stable Beluga series. These models represented Stability AI's investment in advancing open-source LLM capabilities beyond their better-known Stable Diffusion image generation work. The Stable Beluga releases contributed important research findings about efficient fine-tuning methodologies that have been widely replicated and extended by the open-source AI community.