LiteLLM

Route Fabro models through a LiteLLM proxy

LiteLLM can run as an OpenAI-compatible proxy in front of many model providers. Fabro includes a disabled litellm provider entry so you can opt in from settings.toml without changing Fabro code.

Prerequisites

• A running LiteLLM proxy reachable from the Fabro process
• At least one LiteLLM model name you want Fabro to route to
• A LiteLLM key or placeholder key available to Fabro

Fabro’s built-in LiteLLM provider points at http://localhost:4000/v1. Change base_url if your proxy is hosted elsewhere.

Enable the provider

Add the provider override and one or more model entries to ~/.fabro/settings.toml:

settings.toml

_version = 1

[llm.providers.litellm]
enabled = true
base_url = "http://localhost:4000/v1"

[llm.providers.litellm.models."litellm-gpt-5"]
api_id = "gpt-5"
display_name = "LiteLLM GPT-5"
family = "litellm"
default = true

[llm.providers.litellm.models."litellm-gpt-5".limits]
context_window = 128000
max_output = 8192

[llm.providers.litellm.models."litellm-gpt-5".features]
tools = true
vision = false
reasoning = false

api_id is the model name Fabro sends to LiteLLM. It should match a model name configured in your LiteLLM proxy.

Configure credentials

For server-backed runs, store LITELLM_API_KEY in the Fabro server vault.

For a server-owned secret:

fabro secret set LITELLM_API_KEY sk-proxy-key

fabro exec and direct fabro-llm SDK usage can use an env-backed credential source explicitly:

export LITELLM_API_KEY=sk-proxy-key

If your local LiteLLM proxy does not enforce authentication, use a placeholder value such as anything; the OpenAI-compatible client still needs a credential value.

Use LiteLLM models

Once the provider is enabled and at least one model is declared, use the Fabro model ID like any other catalog model:

fabro model list --provider litellm
fabro model test --model litellm-gpt-5
fabro run workflow.fabro --model litellm-gpt-5

In workflow stylesheets:

workflow.fabro

digraph Example {
    graph [
        model_stylesheet="
            * { model: litellm-gpt-5; }
        "
    ]

    start [shape=Mdiamond, label="Start"]
    work  [label="Work", prompt="Use the configured LiteLLM model."]
    exit  [shape=Msquare, label="Exit"]

    start -> work -> exit
}

Declaring more models

Declare each LiteLLM-routed model explicitly so Fabro knows its provider, context window, tool support, and routing defaults:

settings.toml

[llm.providers.litellm.models."litellm-fast"]
api_id = "fast-model"
display_name = "LiteLLM Fast"
family = "litellm"
aliases = ["fast"]

[llm.providers.litellm.models."litellm-fast".limits]
context_window = 64000
max_output = 4096

[llm.providers.litellm.models."litellm-fast".features]
tools = true
vision = false
reasoning = false

Only one model for a provider should set default = true. You may also mark one small/cheap utility model with small_default = true; Fabro uses it for metadata tasks such as generated run titles and falls back to the provider default when it is omitted.

Troubleshooting

“No API key configured” — For runs, set vault:LITELLM_API_KEY with fabro secret set LITELLM_API_KEY .... Exporting it in the server’s shell has no effect on runs: workers start from a cleared environment and provider keys are not inherited. For fabro exec or direct SDK usage, export LITELLM_API_KEY in the invoking shell and use an env-backed credential source.

Connection refused — Confirm the LiteLLM proxy is running and that base_url is reachable from the Fabro process. For Docker deployments, localhost means the Fabro container unless you point it at a host or service name.

Unknown model from LiteLLM — Check that the model’s api_id matches the model name configured in LiteLLM, then run fabro model test --model <fabro-model-id>.

Further reading

Models

How Fabro routes model IDs, providers, and fallbacks.

Settings Configuration

Full reference for provider settings and provider-scoped model offerings.