What Are Distilled AI Models? A Look at LLM Distillation and Its Outputs

Yúbal Fernández

Writer

This article breaks down distilled AI models in a simple, easy-to-understand way. In our guide on downloading and installing DeepSeek on your computer, we mention distilled versions of this model and other systems.

LLM distillation refers to large language models (LLMs) capable of processing text, understanding input, and generating responses. Examples include ChatGPT, DeepSeek, Copilot, Gemini, and Grok.

What’s LLM distillation?

Distilling AI models is a technique that reduces model size while maintaining performance.

LLMs require significant storage and computing power. When you use an AI model through a website or app, you’re connecting to the company’s servers, where the model runs. However, installing a full model on your computer requires a powerful processor and ample storage.

A distilled AI model solves this problem by taking up less space while replicating most of the original model’s performance. These models run faster and require fewer resources.

The process works like a teacher-student relationship. The original, full-scale model (teacher) trains a smaller version (student) by transferring knowledge and experience. The student model learns to mimic the teacher’s abilities in a more compact, efficient way.

The result is a lighter model. Although it won’t match the teacher model’s accuracy, it retains core features and functionality, making it a more versatile, streamlined version.

There are different techniques for creating distilled models. Some involve knowledge distillation, where the student model learns from the teacher’s final outputs. Others use intermediate layers to transfer decision-making processes. Some methods even involve multiple teacher models to enhance training.

Tech companies typically create distilled versions of their models. These versions often get special names, such as Google Gemini’s “Flash” or OpenAI’s “Mini.”

For open source models, the distilled version’s name may include the teacher model and student model. For example, a distilled version of DeepSeek trained on Qwen might be called DeepSeek Qwen or DeepSeek Distill Qwen.

Pros and Cons of Distilled AI Models

A full AI model contains billions of parameters, requiring massive computing power and storage. Running it on a home computer demands the latest technology, while companies like OpenAI and Google need extensive server resources to offer their models via web or app.

Distilled models reduce size and improve efficiency. They run faster and require less computation, allowing companies to provide free, lightweight versions while reserving full models for paying users. Maintaining full-scale models requires significant investment.

For open source AI, distilled versions let users install and run models on personal computers without expensive hardware upgrades. Distillation also lowers the cost of creating AI systems by training new models on existing ones rather than starting from scratch.

However, distilled models have fewer parameters, making them less capable. They tend to generate more errors and hallucinations.

For example, if you install DeepSeek, you’ll notice different versions: 8B, 14B, and the full 671B model. The number represents parameters—the smaller the number, the fewer resources needed, but the more distilled the model is.

A DeepSeek 8B model will have more hallucinations and provide less accurate answers than a 14B version. Similarly, commercial models follow the same pattern. Gemini 2.0 Flash won’t perform as well as the full Gemini 2.0, just as OpenAI’s o3 Mini is less powerful than o3. However, companies offer Flash or Mini versions for free while charging for full models to cover operational costs.

Image | Xataka On with Grok

Related | How to Use DeepSeek: 36 Features and Tricks to Get the Most Out of This AI Model

Related | Goku AI: Breaking Down ByteDance’s New Open Source AI Model for Creating Videos