Skip to main content

​
What NagaAI does with each request

Most models in the catalog are served by several providers. For every chat request NagaAI:
  1. Takes the providers that serve the requested model and are healthy right now.
  2. Orders them by the routing strategy.
  3. Sends the request to the first one. If that provider fails with a timeout, a 5xx, a rate limit or an exhausted quota, NagaAI tries the next one, up to three attempts. NagaAI does not retry a 400 or 422 from a provider.
You get one answer or one error. The response never names the provider, and NagaAI does not charge failed attempts. The model field of the response is the NagaAI catalog id, whichever provider answered. Because the provider can change from one request to the next, two identical requests can land on different providers. Latency, cache hits and exact wording can differ between them.

​
Strategies

A strategy changes the order in which NagaAI tries providers. It never removes a provider and never makes a request fail. If you do not choose a strategy, NagaAI chooses one, and the response header below tells you which. Strategies apply to /v1/chat/completions, /v1/responses and /v1/messages. Images, audio, embeddings and moderation ignore them.

​
Choosing a strategy

There are three places to set one. When more than one is present, the first in this list wins.
  1. The request body: a top-level routing object.
  2. The x-naga-routing-strategy request header.
  3. The API key’s default, set through the API keys API.

​
In the request body

The same routing object works in all three chat formats. The value is case-sensitive. An unknown strategy or an unknown key inside routing fails with 400 invalid_request_error:
SDKs pass extra body fields through their own option:

​
In a header

Use the header when you cannot change the body, for example in a tool that only lets you add headers.
The header is case-insensitive. NagaAI ignores an unknown value in the header instead of failing the request.

​
As a key default

Set default_routing_strategy when you create or update a key through the account API. Every request made with that key uses the strategy unless the body or the header names another. Send null in an update to go back to letting NagaAI choose.

​
Checking which strategy ran

Every successful chat response carries the strategy NagaAI used:
Errors returned before NagaAI reaches a provider do not carry this header.

​
What you cannot choose

NagaAI does not let a request name a provider, exclude one, or list fallback models. To use another model after a failure, catch the error and send a new request with that model.