Distillable Models and Synthetic Data Pipelines with NeMo Data Designer

1. Introduction to Distillable Models

In the rapidly evolving landscape of artificial intelligence, model distillation has emerged as a crucial technique for deploying high-performance models in resource-constrained environments. Distillable models allow developers to transfer the knowledge of large, complex models into smaller, more efficient ones without significant loss in performance.

2. NeMo Data Designer Overview

NeMo Data Designer is NVIDIA's latest framework for creating synthetic data pipelines that power model distillation. It provides a unified interface for generating high-quality training data, managing data quality metrics, and orchestrating the entire distillation workflow from start to finish.

Key Features

The framework includes automated data generation with configurable quality filters, support for multi-modal data pipelines, built-in evaluation metrics for synthetic data quality, and seamless integration with existing NeMo training workflows. These features make it possible to create production-ready distilled models in a fraction of the time previously required.

3. Getting Started with PP API

PP API now supports distillable models through our unified interface. You can access NeMo-distilled models with the same API calls you use for any other model, with automatic routing to the most cost-effective provider. Our platform handles the complexity of model selection and routing, so you can focus on building your application.

import requests

response = requests.post(
    "https://api.ppapi.ai/v1/chat/completions",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "nemo-distilled-v2",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain model distillation."},
        ],
        "temperature": 0.7,
    },
)

print(response.json())