Routing

Most models in the catalog are served by several providers. NagaAI picks one for every chat request and moves the request to another when the first fails. An outage at one provider therefore doesn’t reach your app. A routing strategy decides which provider NagaAI tries first.

Provider order and retries

For every chat request NagaAI goes through these steps:

Takes the providers that serve the requested model and are healthy right now.
Orders them by the routing strategy.
Sends the request to the first one. On a timeout, a 5xx, a rate limit or another provider-side failure, NagaAI tries the next provider, up to three attempts. A 400 or 422 from a provider isn’t retried.

You get one answer or one error. The response never names the provider, and its model field holds the NagaAI catalog id, whichever provider answered. Attempts that failed before the answer cost nothing, see Billing. The provider can change from one request to the next. Two identical requests can therefore differ in latency, cache hits and exact wording.

Strategies

A strategy changes the order in which NagaAI tries providers. It never removes a provider and never makes a request fail. Each strategy puts a different provider first:

Strategy	Prefers
`reliability`	The provider with the best recent success rate
`balanced`	Speed, among providers whose success rate is close to the best
`latency`	The shortest time to the first token. Without streaming, the fastest complete answer
`throughput`	The most output tokens per second
`cache`	Providers that return prompt-cache reads for this model, which lowers the price of long repeated prompts

Without a strategy in the request, NagaAI chooses one and names it in the response header described under Check which strategy ran. Strategies apply to /v1/chat/completions, /v1/responses and /v1/messages. Image, audio, embedding and moderation requests ignore them.

Set a strategy

You can set a strategy in the request body, in a header or as a default on the API key. When a request has more than one, the body wins over the header, and the header wins over the key default.

In the request body

Add a top-level routing object. It works the same way in all three chat formats:

import os

from openai import OpenAI

client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])

response = client.responses.create(
    model="anthropic/claude-sonnet-5.5",
    input="Suggest a name for a Python library that parses invoices.",
    extra_body={"routing": {"strategy": "latency"}},
)

print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.naga.ac/v1", apiKey: process.env.NAGA_API_KEY });

const response = await client.responses.create({
  model: "anthropic/claude-sonnet-5.5",
  input: "Suggest a name for a Python library that parses invoices.",
  routing: { strategy: "latency" },
});

console.log(response.output_text);

curl https://api.naga.ac/v1/responses \
  -H "Authorization: Bearer $NAGA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5.5",
    "input": "Suggest a name for a Python library that parses invoices.",
    "routing": {"strategy": "latency"}
  }'

The Python SDK sends fields it doesn’t know through extra_body. The value is case-sensitive. An unknown strategy or an unknown key inside routing fails with 400 invalid_request_error:

{"error": {"type": "invalid_request_error", "message": "body.routing.strategy: unknown variant `fast`, expected one of `reliability`, `balanced`, `latency`, `throughput`, `cache` at line 1 column 114"}}

In a header

Use the header when you can’t change the body, for example in a tool that only lets you add headers:

x-naga-routing-strategy: throughput

The header is case-insensitive. An unknown value in the header doesn’t fail the request: NagaAI ignores it and chooses a strategy itself.

As a key default

Set default_routing_strategy when you create or update a key through the API keys API. Every request made with that key then uses the strategy, unless its body or header names another. Send null in an update to let NagaAI choose again.

curl -X PATCH https://api.naga.ac/v1/account/keys/KEY_ID \
  -H "Authorization: Bearer $NAGA_PROVISIONING_KEY" \
  -H "Content-Type: application/json" \
  -d '{"default_routing_strategy": "cache"}'

Check which strategy ran

Every successful chat response carries the strategy NagaAI used in a header:

x-naga-routing-strategy: latency

An error returned before NagaAI reaches a provider has no such header.

Fallback models

When the model you asked for can’t answer, NagaAI can send the same request to another model. List the fallbacks in the order to try them: models in Responses and Chat Completions, fallbacks in Messages, up to three. The fields have the shapes OpenRouter and Anthropic use, so code written for them works unchanged.

import os

from openai import OpenAI

client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])

response = client.responses.create(
    model="anthropic/claude-sonnet-5.5",
    input="Suggest a name for a Python library that parses invoices.",
    extra_body={"models": ["anthropic/claude-haiku-5.5"]},
)

print(response.model)
print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.naga.ac/v1", apiKey: process.env.NAGA_API_KEY });

const response = await client.responses.create({
  model: "anthropic/claude-sonnet-5.5",
  input: "Suggest a name for a Python library that parses invoices.",
  models: ["anthropic/claude-haiku-5.5"],
});

console.log(response.model);
console.log(response.output_text);

curl https://api.naga.ac/v1/responses \
  -H "Authorization: Bearer $NAGA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5.5",
    "input": "Suggest a name for a Python library that parses invoices.",
    "models": ["anthropic/claude-haiku-5.5"]
  }'

The model field of the answer names the model that answered: anthropic/claude-sonnet-5.5 when it could, anthropic/claude-haiku-5.5 when it couldn’t. Chat Completions takes models the same way. In Messages, each fallback is an object with a model key, as in Anthropic’s API:

Python

import os

from anthropic import Anthropic

client = Anthropic(base_url="https://api.naga.ac", api_key=os.environ["NAGA_API_KEY"])

message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=512,
    messages=[{"role": "user", "content": "Suggest a name for a Python library that parses invoices."}],
    extra_body={"fallbacks": [{"model": "anthropic/claude-haiku-5.5"}]},
)

print(message.model)
print(message.content[0].text)

When NagaAI moves to the next model

Each model first gets the provider retries described in Provider order and retries. NagaAI moves to the next model in the list when:

every provider of the model failed, or none can serve it right now;
the provider’s moderation refused the request;
the model can’t take the request, for example an input type it doesn’t accept.

Some failures would repeat on every model, so NagaAI stops at them. Such failures are a body that doesn’t parse, a tool call without its result, an empty balance and a rate limit. It also doesn’t switch once an answer has started to stream. When every model in the list fails, you get the error of the last one.

Price

You pay only for the model that answered, at that model’s price. A fallback can cost more than the model you asked for, so list only models whose price suits you. Attempts that failed cost nothing. Without credits on your balance, NagaAI skips paid fallbacks behind a :free model and still runs the free one. A :free fallback counts against the free-model rate limit; when you hit the limit, NagaAI skips it.

Errors

A fallback that isn’t in the catalog fails the whole request with 400 model_not_found, the same way the model itself would. More than three fallbacks fail with 400 invalid_request_error. NagaAI tries a model that appears twice only once.

Images

POST /v1/images/generations and POST /v1/images/edits take models the same way, with the same limit of three and the same rules for when NagaAI moves on and what you pay. A model that can’t take the request, such as an n above its max_images, is skipped before NagaAI moderates the prompt. Image generation has the example.

Video

POST /v1/videos takes models the same way. A model that doesn’t sell the clip you asked for, such as its duration or resolution, is skipped before NagaAI moderates the prompt. The job belongs to the model whose provider accepted it, and you pay that model’s price per second. Video generation has the example.

Embeddings, speech, transcription and moderation

POST /v1/embeddings, POST /v1/audio/speech, POST /v1/audio/transcriptions, POST /v1/audio/translations and POST /v1/moderations take models the same way. The two audio upload routes take it as form parts, one models part per model. Speech moves on only before the first byte of audio. Each guide says what to check when you pick fallbacks: embeddings, text to speech, speech to text and moderation.

Provider choice

A request can’t name a provider or exclude one. NagaAI picks among the providers of each model as described above.

Get started

How NagaAI works

Guides

API guides

Account

Product

Provider order and retries

Strategies

Set a strategy

In the request body

In a header

As a key default

Check which strategy ran

Fallback models

When NagaAI moves to the next model

Price

Errors

Images

Video

Embeddings, speech, transcription and moderation

Provider choice

​Provider order and retries

​Strategies

​Set a strategy

​In the request body

​In a header

​As a key default

​Check which strategy ran

​Fallback models

​When NagaAI moves to the next model

​Price

​Errors

​Images

​Video

​Embeddings, speech, transcription and moderation

​Provider choice

Provider order and retries

Strategies

Set a strategy

In the request body

In a header

As a key default

Check which strategy ran

Fallback models

When NagaAI moves to the next model

Price

Errors

Images

Video

Embeddings, speech, transcription and moderation

Provider choice