Usage

Once you have nilAI API access, you can start using LLMs on nilAI nodes with any OpenAI-compatible library.

info

The Old Flow will be deprecated by the 1st of August 2025. Please use the NUCS flow to create your API key.

NUCS

Getting Started with Private LLMs

Python

pip

pip install nilai-py

uv

uv pip install nilai-py

TypeScript

pnpm install @nillion/nilai-ts

You can either use:

• API Key flow as the sole developer / organization or
• Delegation Flow to provide permissions to another user / organization.

API Key Flow

1. Use https://nilai-a779.nillion.network/nuc/v1 as the BASE URL
2. Check available models or query the /v1/models endpoint or
3. Select an available model and use it with the /v1/chat/completions nilAI node endpoint

With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B model:

Python

examples/0-api_key_mode.py

loading...

View on GitHub

Typescript

import "dotenv/config";
import { NilaiOpenAIClient, NilAuthInstance } from "@nillion/nilai-ts";

// To obtain an API key, navigate to https://subscription.nillion.com
// and create a new subscription.
// The API key will be displayed in the subscription details.
// The NilaiOpenAIClient class automatically handles the NUC token creation and management.

const API_KEY = process.env.NILLION_API_KEY;

async function main() {
// Initialize the client in API key mode
// For sandbox, use the following:
const client = new NilaiOpenAIClient({
    baseURL: "https://nilai-a779.nillion.network/nuc/v1/",
    apiKey: API_KEY,
    nilauthInstance: NilAuthInstance.SANDBOX,
    // For production, use the following:
    // nilauthInstance: NilAuthInstance.PRODUCTION,
});

// Make a request to the Nilai API
const response = await client.chat.completions.create({
    model: "google/gemma-3-27b-it",
    messages: [
    { role: "user", content: "Hello! Can you help me with something?" }
    ],
});

console.log(`Response: ${response.choices[0].message.content}`);
}

// Run the example
main().catch(console.error);

Delegation flow

To use the delegation flow, you need to create a delegation token server.

The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.

Python

examples/1-delegation_token_mode.py

loading...

View on GitHub

Typescript

examples/1-delegation-token.ts

loading...

View on GitHub

Old Flow

Getting Started with SecretLLM

Getting started with SecretLLM is straightforward:

1. Select a nilAI node url
2. Query the /v1/models endpoint or check available models
3. Select an available model and use it with the /v1/chat/completions nilAI node endpoint

Since SecretLLM is OpenAI-compatible, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B model:

Node.js

nilai/secretllm_nodejs/index.js

loading...

View on GitHub

Example response from SecretLLM

Signature and Response

Signature: MEYCIQCfafKvn1nRmBdFYyxp+mJrRVaTuuN1ME40TRDC4Wg/HAIhAIpm4H2QCqlPmgnOd8/aJXuvWvIAEzKl3ObbbyFMz6iI
Response: As a fitness coach, I'd say it's not necessarily about which one is better, but rather about balance and moderation. Both salads and pizza can be part of a healthy diet, but it depends on how they're prepared and consumed.

A salad, when made with nutrient-dense ingredients like leafy greens, vegetables, lean proteins, and healthy fats, can be an excellent choice for a healthy meal. It's high in fiber, vitamins, and minerals, and can help support weight loss, improve digestion, and boost energy levels.

On the other hand, pizza, when made with refined flour, processed meats, and high amounts of cheese, can be a less-than-ideal choice. It's high in calories, saturated fat, sodium, and refined carbohydrates, which can lead to weight gain, inflammation, and other health problems.

However, if you're craving pizza, there are ways to make it healthier. Look for options that use:

1. Whole-wheat or cauliflower crust
2. Lean protein sources like chicken, turkey, or plant-based options
3. Fresh vegetables like bell peppers, onions, and mushrooms
4. Low-fat cheese or dairy-free alternatives
5. Herbs and spices for flavor instead of salt and sugar

In contrast, a salad can be made healthier by:

1. Using a variety of colorful vegetables
2. Adding lean protein sources like grilled chicken, salmon, or tofu
3. Incorporating healthy fats like avocado, nuts, or seeds
4. Using a homemade vinaigrette dressing instead of store-bought
5. Limiting or avoiding added sugars and refined grains

Ultimately, the better choice between salad and pizza depends on your individual needs and preferences. If you're looking for a quick, easy, and nutritious meal, a salad might be the way to go. But if you're craving something more indulgent, a healthier pizza option can be a better choice.

As a fitness coach, I always recommend balance and variety in your diet. Aim to include a mix of whole, unprocessed foods, and indulge in moderation. Listen to your body and make choices that nourish your mind, body, and soul.

Python

from openai import OpenAI

# Initialize the OpenAI client

client = OpenAI(
    base_url="https://nilai-<node>.nillion.network/v1/",
    api_key="YOUR_API_KEY"
)

# Send a chat completion request

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is your name?"
        }
    ],
    stream=False
)

Next JS (API routes)

nilai/secretllm_nextjs/app/api/chat/route.ts

loading...

View on GitHub