Models & Providers

Ragz never talks to an LLM provider directly. Every chat completion and every embedding call goes through the bundled LiteLLM gateway (RAGZ_LITELLM_URL), which is what lets you mix providers — a cloud model for generation, a local model for embeddings, Cohere for reranking — behind one consistent interface, with per-user/per-org spend tracked centrally.

Two separate registries

Admin → Models manages chat and embedding models (which providers, which credentials, which capabilities). Admin → Settings manages the provider choice for parsing, reranking, and web search. They're independent: adding an OpenAI model doesn't change your parser, and switching parsers doesn't touch your model list.

Adding a model

From Admin → Models, a superadmin adds a model with:

Field	Meaning
`litellm_model_name`	The model name passed to LiteLLM verbatim — e.g. `gpt-4o`, or `gemini/gemini-2.5-pro` for non-OpenAI providers (the catalog prefix is required).
`display_name`	What users see in the model picker.
`provider_kind`	`openai` \| `litellm` (any other LiteLLM-native provider — Anthropic, Gemini, Groq, ...) \| `ollama` \| `openai_compatible` \| `tei` (reserved for the built-in local embedder, not creatable here).
`base_url`	Required for `ollama` and `openai_compatible`; the endpoint of your self-hosted server.
`api_key`	Write-only — stored encrypted, never echoed back.
`modality`	`chat` (default) or `embedding`. Embedding models require `dimension`; chat models must not set it.
`supports_reasoning` / `default_reasoning_effort`	Whether the model accepts a reasoning-effort parameter (`off`/`low`/`medium`/`high`) and what to default it to.
`supports_vision`	Whether the model accepts image input.
`tools_unreliable`	Flags a model as unreliable at native tool-calling — the agent loop falls back to a JSON-planner protocol instead.

curl -X POST https://your-ragz-host/api/v1/admin/models \
  -H "Authorization: Bearer $SUPERADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "litellm_model_name": "gpt-4o",
    "display_name": "GPT-4o",
    "provider_kind": "openai",
    "api_key": "sk-...",
    "supports_vision": true,
    "supports_reasoning": false
  }'

Once added, a model becomes selectable as the workspace's generation model (the chat model picker) or, for modality: "embedding", as a workspace's embedding model.

Generation model

Any enabled modality: "chat" model can be selected per workspace or per conversation from the chat model picker. Users only see display_name, supports_reasoning/default_reasoning_effort, and supports_vision — not provider details or keys.

Embeddings

Ragz ships a built-in local embedder — bge-m3 served by a TEI (Text Embeddings Inference) container, RAGZ_TEI_URL, 1024-dimensional (RAGZ_EMBEDDING_DIM). This is the provider_kind: "tei" row that's bootstrap-seeded and can't be created through the API.

Alternatively, add an API embedding model (provider_kind: "openai" or "litellm", modality: "embedding", with dimension set to match the provider's output size — e.g. 1536 for text-embedding-3-small).

Embedding model is locked per workspace

A workspace's embedding model is fixed at first use and can't be swapped afterward — every document in the workspace is embedded into a Qdrant collection sized for that model's dimension (collection_name in ModelOut). Changing embedding models means creating a new workspace and re-ingesting.

Reranker

The reranker reorders retrieval candidates after the vector search, before they reach the LLM. Choose it in Admin → Settings → Reranker:

Option	Where it runs	Setup
local (default)	Self-hosted TEI cross-encoder, `RAGZ_RERANK_URL`	No key needed
cohere	Cohere's hosted rerank API	Requires a Cohere API key (Admin → Settings); model defaults to `rerank-v4.0-fast`, or `rerank-v4.0-pro`

Cloud vs. local / air-gapped placement

Everything except the model servers themselves idles at roughly 1–1.5 GB. What you place locally vs. call over an API is what drives RAM:

Setup	Local model servers	Host RAM
Cloud — API embeddings (OpenAI) + API rerank (Cohere)	none	4–5 GB
Hybrid — one local (embeddings or rerank)	one	8–10 GB
Fully local / air-gapped — local `bge-m3` + local rerank	two	12–16 GB

The generation LLM sits outside this table: a cloud API needs no GPU; a local LLM (Ollama/vLLM) needs a 16 GB+ GPU. For a fully air-gapped install, set RAGZ_MODEL_CATALOG_URL= (empty) so the pricing/context-window catalog sync never makes a network call, and point every model at a litellm/ollama provider with no outbound dependency.