Migration Bifrost -> LiteLLM

development

Author

Werner Schiller

Published

September 28, 2026

A few words on my migration from Bifrost to LiteLLM.

🤔 Why ?

My main reason for Bifrost initially was that it offered a good MCP proxy. Meanwhile, LiteLLM also includes that. That alone is not enough, so the main reasons why I switched are:

curiosity (that could be a main theme … 😉)
a lot of interesting features of Bifrost are behind a paywall / enterprise payment.
LiteLLM seems to be more broadly used than Bifrost

👨‍🏭 How ?

First I let an AI (I think it was GLM 5.3 Flash) extract the Bifrost configuration and generate the config JSON. I had to clean that up and decide how to define it properly. In the end I discarded a lot of the migration and manually defined some setup. Disclaimer: I did NOT yet bake the config into an Ansible configuration; I’ll do that after a testing phase.

🚧 Installation

I have a rules.md that defines a specific layout for new services on my main homelab server, which I used to create a compose file and directory layout. Afterwards it just ran. It is specific to my setup, so I post a much simpler version here:

services:
  litellm-db:
    image: postgres:16
    container_name: litellm-db
    restart: unless-stopped
    env_file:
      - .env
    volumes:
      - ./data/db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 15s
    networks:
      - litellm-internal

  litellm:
    restart: unless-stopped
    image: ghcr.io/berriai/litellm:main-stable
    container_name: litellm
    env_file:
      - .env
    environment:
      SSL_CERT_FILE: /etc/ssl/local/my.local.crt
    depends_on:
      litellm-db:
        condition: service_healthy
    volumes:
      - ./data:/app/data
      - ./config.yaml:/app/config.yaml:ro
      - ./my.local.crt:/etc/ssl/local/my.local.crt:ro
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness', timeout=10)"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    networks:
      - litellm-internal

networks:
  litellm-internal:

with the .env file:

LITELLM_MASTER_KEY=<< generated master key >>

OPENROUTER_API_KEY=<< openrouter-api-key >>
DEEPSEEK_API_KEY=<< deepseek-api-key >>
HETZNER_API_KEY=<< hetzner-api-key >>
ZAI_API_KEY=<< zai-api-key >>
XIAOMI_API_KEY=<< xiaomi-api-key >>
OPENCODE_API_KEY=<< opencode-api-key >>
CONTEXT7_API_KEY=<< context7-api-key >>
HA_MCP_TOKEN=<< homeassistant-mcp-token >>
POSTGRES_USER=litellm
POSTGRES_PASSWORD=<< generated-postgresql-password >>
POSTGRES_DB=litellm
DATABASE_URL=postgresql://litellm:<< generated-postgresql-password >>@litellm-db:5432/litellm

🪏 Extraction

Well, the extraction went relatively smoothly, but the result was a mess. OK - not exactly a mess, but not really usable. I did not yet have a machine-readable list of providers and models I use ready, so I let it extract all the configured models and read the logs to find out which models I actually use.

That went well, but “configured” was only a subset (because I left the models mostly untouched and experimented with aliases - and those aliases were what it thought were the “configured” models) and in the end it extracted all models that were available from my configured providers. As OpenRouter was one of them, the list was huge.

In the end I wrote the whole configuration mostly by hand, but that gave me the opportunity to include the list in my AI-snippets repository. Things that got imported but were pretty useless in the end:

providers
models
virtual keys with their configurations

🎓 Configuration

I wanted to have the configuration as a file, not via UI, so I later have an easier life when I want to put it to Ansible. I am not fully there, but at least the providers, models and MCP servers I managed to put in the config file.

Models / Providers

One feature I really wanted to get was automatic fallback / fallthrough.

As you probably already guessed, I have a few accounts, most notably OpenCode Go, Xiaomi MiMo TokenPlan, Z.ai Coding plan (they have hilarious German translations!) and OpenRouter (pay-as-you-go) as a fallback. For each of the models, I wanted the fallback chain to be something like main provider (optional) -> opencode go -> OpenRouter.

This will trigger when the credits of one provider are empty - the request will then fail and fall back to the next.

After some research I found out that the correct way - and this was hard to find - is to name the same models of different providers internally the same and give them an order number:

model_list:
# DeepSeek V4 Pro
  - model_name: deepseek-v4-pro
    litellm_params:
      model: openai/deepseek-v4-pro
      api_key: os.environ/OPENCODE_API_KEY
      extra_headers:
        x-opencode-session: << some random generated GUID >>
      api_base: https://opencode.ai/zen/go/v1
      order: 1
  - model_name: deepseek-v4-pro
    litellm_params:
      model: deepseek/deepseek-v4-pro
      api_key: os.environ/DEEPSEEK_API_KEY
      order: 2
  - model_name: deepseek-v4-pro
    litellm_params:
      model: openrouter/deepseek/deepseek-v4-pro-0813
      api_key: os.environ/OPENROUTER_API_KEY
      order: 3
# MiMo v2.6 Flash
  - model_name: mimo-v2.6-flash
    litellm_params:
      model: openai/mimo-v2.6-flash
      api_key: os.environ/XIAOMI_API_KEY
      api_base: https://token-plan-ams.xiaomimimo.com/v1
      order: 1
  - model_name: mimo-v2.6-flash
    litellm_params:
      model: openai/mimo-v2.6-flash
      api_key: os.environ/OPENCODE_API_KEY
      extra_headers:
        x-opencode-session: << some random generated GUID >>
      api_base: https://opencode.ai/zen/go/v1
      order: 2
  - model_name: mimo-v2.6-flash
    litellm_params:
      model: openrouter/xiaomi/mimo-v2.6-flash
      api_key: os.environ/OPENROUTER_API_KEY
      order: 3
# Minimax-m3
  - model_name: minimax-m3
    litellm_params:
      model: openai/minimax-m3
      api_key: os.environ/OPENCODE_API_KEY
      extra_headers:
        x-opencode-session: << some random generated GUID >>
      api_base: https://opencode.ai/zen/go/v1
      order: 1
  - model_name: minimax-m3
    litellm_params:
      model: openrouter/minimax/minimax-m3
      api_key: os.environ/OPENROUTER_API_KEY
      order: 2
# Hetzner Experiments
  - model_name: Qwen3.8-27B
    litellm_params:
      model: openai/Qwen3.8-27B
      api_key: os.environ/HETZNER_API_KEY
      api_base: https://inference.hetzner.com/api/v1

# local inference
  - model_name: qwen3-embedding:4b
    litellm_params:
      model: openai/qwen3-embedding:4b
      api_key: << my-internal-ollama-api-key >>
      api_base: https://<< my.ollama.instance >>/v1

The notation was not so intuitive:

field	description
model_name	The internal model name. The same for the same model, so the model only appears once for the clients.
model	The external model name, the first path-section is the LiteLLM identifier for the protocol. `openrouter`, for example, tells LiteLLM exactly where it should connect to and `deepseek/deepseek-v4-pro-0813` is the OpenRouter-specific model name. Similarly, `openai` tells LiteLLM to use the OpenAI protocol, but `api_base` tells it to use a different endpoint to connect, so this is used to connect to OpenAI-compatible endpoints like OpenCode.
api_base	Can be omitted if the provider is the official provider.
order	The order in which LiteLLM attempts to send the request to each provider.
extra_headers	Yeah … this popped up a lot lately; OpenCode now has a mandatory header field and not all clients support that. I think I have to write a plugin for that.

MCPs

No surprises there. I did not yet try to run a stdio -> SSE conversion, but I guess that will come soon. I will update this blog entry then.

mcp_servers:
  context7:
    url: "https://mcp.context7.com/mcp"
    transport: http
    auth_type: none
    static_headers:
      CONTEXT7_API_KEY: os.environ/CONTEXT7_API_KEY
  homeassistant:
    url: "http://<< home-assistant-host-and-port >>/api/mcp"
    transport: http
    auth_type: bearer_token
    auth_value: os.environ/HA_MCP_TOKEN
  forgejo:
    url: "http://<< forgejo-host-and-port >>/mcp"
    transport: http
    auth_type: none

(I have some more MCP servers configured, but they are of no interest here)

Users / Teams / Virtual Keys / etc

Those are things that I managed over the UI. I created one team, added a subset of the configured models and the list of allowed MCP servers there and then let the Virtual Keys be part of that team. This way the clients only see exactly the models that I want them to see - and I can restrict the model list per key even further. For the Virtual Keys I kept the idea of one key per application. That means OpenCode, ZooCode, Home Assistant, OpenWebUI all have their own keys and I can monitor and manage each application individually.

🌩️ Problems

As mentioned above, the main problem was that I was not prepared properly with a predefined list of models that I use and the AI export from Bifrost was suboptimal. That probably shows that - obviously - any AI can’t read your mind, is only as good as the input it has and sometimes it’s more efficient to just do it yourself than to let the AI knit some second-hand suboptimal sweater that you then have to manually unravel and knit again for it to be acceptable.

The second problem was to find a solution for how to hide the external model names from the clients. Aliases, mapping, basically anything that I found in the GUI just did not do the job. As mentioned above, the solution was to use the same internal model_name and add order for the same models on different providers.

🏁 Future topics

I only scratched the surface here and LiteLLM has a lot more features to offer, even in the Community version.

Future topics are:

logging, enable request logging
SSO login
alerting
guardrails / policies
cost optimization
vector stores
memory
search tools
skills
A2A

I will create separate posts when I get to those topics.