Skip to content
Yttri Docs

Your own gateway for multiple models

A “custom provider” in Yttri is any server that answers an OpenAI-compatible /chat/completions. That opens a third option besides the cloud and a single local model: your own gateway that decides where each request goes — to different cloud providers, to several local models, or both — and hands Yttri one single entry point.

Tools like this go by different names — router, proxy, model aggregator. The best known and best documented is LiteLLM; the example below uses it, but the principle is the same for any tool in this category.

LiteLLM in Docker

Section titled “LiteLLM in Docker”

A config.yaml — the list of models and where each one goes:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: local-qwen
    litellm_params:
      model: ollama/qwen2.5
      api_base: http://host.docker.internal:11434

Run it:

Окно терминала
docker run -d \
  -p 4000:4000 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  -e OPENAI_API_KEY=sk-... \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  ghcr.io/berriai/litellm:main-stable \
  --config /app/config.yaml

Connecting to Yttri

Section titled “Connecting to Yttri”

Settings → Assistant → Models → Add provider:

Field Value
API type Chat Completions (OpenAI-compatible)
Base URL http://localhost:4000
API key the key you set in LiteLLM (master_key in the config), if auth is enabled

The model list you’ll see afterwards is the model_name values from your config.yaml (gpt-4o, claude, local-qwen in the example above), not the providers’ original model names.