AI providers and models

A source is a provider connection: URL, credential and available models. You may keep several sources, choose a deployment default, then select another model for a conversation. Embeddings are configured separately.

Where to configure it

Single-user: Settings → AI provider. Shared mode: Admin → AI provider, available to the global administrator. Members select only the models their permissions allow; they do not set deployment API keys in their personal Settings.

Add and connect a source

• + Add source creates a source editor. Set a useful Source name, such as Local Ollama or Shared vLLM.
• Provider selects OpenAI, Anthropic, Ollama or Other compatible. The latter covers vLLM, SGLang, llama.cpp and gateways implementing the same API contract.
• Ollama uses a root such as http://ollama:11434 without /v1. Other compatible uses a root such as http://host-gateway:8000/v1. OpenAI and Anthropic supply their official roots.
• Enter the API key if required and click Connect. It retrieves the model list or shows a connection error; connecting alone does not save a source.
• Choose a model to save the form automatically. If missing from the list, open Model not listed? Enter its ID, then Save model ID. Use the exact served identifier.

Default model, saved state and credentials

Use as default saves the chosen source/model as the default for new conversations. The source list identifies the current default. Existing sources remain available in the chat picker. Save other changes persists source names and embedding options; wait for All changes saved before leaving.

A blank key preserves the saved credential only for the same connection. Remove saved key explicitly clears it on save. Remove source discards an unsaved source immediately; saved removals ask for confirmation and must preserve a usable default. Use server settings restores environment defaults when available.

Per-model context and image support

Open Model settings… below the model selector. Context window → Automatic uses reported metadata; Set token limit records the effective server context in tokens, for example 500000 only if the serving configuration really permits it. This does not enlarge the provider’s context window.

Vision support is Automatic, Supports images or Text only. The UI distinguishes auto-detected, manual and not detected. Missing/null metadata does not prove the model is text-only. Use the effective metadata of the exact source/model; identical model names on different servers can have different limits. Save in this dialog does not change the default model.

Prism compacts older conversation context near the effective limit while reserving room for instructions, tools and generation. The visible transcript remains. An explicit LIVE_CONTEXT_CHAR_BUDGET is an additional deployment ceiling, not a per-model token limit.

Document search and embeddings

Use same provider reuses the default conversation connection and credential; it does not turn a chat model into an embedding model. Choose a model that implements the embedding endpoint. Uncheck the option to select another configured source or a dedicated connection. Anthropic requires another embedding provider.

Connect retrieves candidates. Test embeddings sends a real request and reports vector dimension; a provider catalogue may mix conversation and embedding models. Leave the embedding model empty to disable document search after saving. Models remain usable for chat without an index.

Change providers without losing the index

When changing an existing embedding model or endpoint, select Rebuild the document index when saving. This sends all indexed text to the new provider and may incur API charges. Save applies the change without restarting Prism; a failed rebuild retains the previous index. Correct the endpoint/key and save again.

Connection and model troubleshooting

• Connection refused: check container reachability, listener binding, DNS and firewall. localhost inside Prism points to Prism, not the Docker host.
• 401/403: verify the key, account permissions and provider URL. A key for one provider is not valid on another.
• Model not found: compare the ID with the server catalogue and served-model alias.
• Tool calls appear as plain text: verify the model supports tools and that your serving stack uses the required chat template and tool-call parser.
• Image rejected: select Text only for that model, or correct the server’s image support. A successful text test is not a vision test.