Stable Beluga 2
by Stability AI
Fine-tuned LLaMA2 70B model by Stability AI for instruction-following tasks
Visit Product
66 upvotes
992 views
About
Stable Beluga 2 is a fine-tuned language model built on Meta's LLaMA2 70B foundation, developed by Stability AI's CarperAI lab. It was trained using an Orca-style synthetic data methodology — generating high-quality instruction-following examples with detailed explanations ("thought chains") from GPT-4 and then using these to fine-tune the base model.
This approach, called "Orca-style training", teaches the model not just to answer questions but to reason through them — producing more thoughtful, accurate, and well-explained responses than standard RLHF fine-tuning alone. Stable Beluga 2 demonstrated state-of-the-art performance among open-source models at its release, competing with and often outperforming larger models on reasoning benchmarks.
The model represents Stability AI's commitment to advancing the open-source LLM ecosystem beyond just image generation, demonstrating that the synthetic data approach originally developed by Microsoft for their Orca models could be effectively applied to open-weight models to dramatically improve their reasoning capabilities.
This approach, called "Orca-style training", teaches the model not just to answer questions but to reason through them — producing more thoughtful, accurate, and well-explained responses than standard RLHF fine-tuning alone. Stable Beluga 2 demonstrated state-of-the-art performance among open-source models at its release, competing with and often outperforming larger models on reasoning benchmarks.
The model represents Stability AI's commitment to advancing the open-source LLM ecosystem beyond just image generation, demonstrating that the synthetic data approach originally developed by Microsoft for their Orca models could be effectively applied to open-weight models to dramatically improve their reasoning capabilities.
Product Features
- Fine-tuned on synthetic Orca-style reasoning data
- Built on LLaMA2 70B for strong baseline capability
- Strong reasoning and instruction-following ability
- Open weights available on Hugging Face
- 70B and smaller distilled versions
- Compatible with standard LLaMA inference tools
- Competitive with larger models on reasoning benchmarks
- Suitable for fine-tuning on specialized tasks
- Available for commercial use within LLaMA2 license terms
- Quantized versions for deployment flexibility
- Built on LLaMA2 70B for strong baseline capability
- Strong reasoning and instruction-following ability
- Open weights available on Hugging Face
- 70B and smaller distilled versions
- Compatible with standard LLaMA inference tools
- Competitive with larger models on reasoning benchmarks
- Suitable for fine-tuning on specialized tasks
- Available for commercial use within LLaMA2 license terms
- Quantized versions for deployment flexibility
About the Publisher
Stability AI is best known for creating Stable Diffusion, the open-source image generation model. Beyond images, the company has been active in developing and releasing open-source language models through its CarperAI research lab. Founded by Emad Mostaque in 2020, Stability AI raised $101 million in 2022 but has faced significant corporate turbulence since. Despite challenges, Stability AI's open-source releases — including the Stable Beluga series — have made important contributions to the LLM ecosystem.