LiteLLM

LiteLLM is the default backend provider, making use of the LiteLLM library to communicate with many LLM providers, as well as local providers and custom APIs.

Chat Backend

The LiteLLM chat backend is enabled by default, using OpenAI's gpt-3.5-turbo model. Adding an OPENAI_API_KEY to your environment is enough to get started using this backend, however the behaviour of this backend can be customised in your Django settings.

To use a specific model, specify the MODEL_ID as below. The names of supported models can be found in the LiteLLM documentation.

WAGTAIL_VECTOR_INDEX = {
    "CHAT_BACKENDS": {
        "default": {
            "CLASS": "wagtail_vector_index.ai_utils.backends.litellm.LiteLLMChatBackend",
            "CONFIG": {
                "MODEL_ID": "gpt-4o",
            },
        },
    },
}

The behaviour of the chat backend can be customised by passing input parameters to LiteLLM. To provide a certain input parameter by default, specify DEFAULT_PARAMETERS in the backend config.

For example; to customise the temperature of all responses:

WAGTAIL_VECTOR_INDEX = {
    "CHAT_BACKENDS": {
        "default": {
            "CLASS": "wagtail_vector_index.ai_utils.backends.litellm.LiteLLMChatBackend",
            "CONFIG": {
                "MODEL_ID": "gpt-4o",
                "DEFAULT_PARAMETERS": {"temperature": 0.8},
            },
        },
    },
}

Embedding Backend

The LiteLLM embedding backend is enabled by default using OpenAI's ada-002 embedding model. Adding an OPENAI_API_KEY to your environment is enough to get started using this backend, however the behaviour of this backend can be customised in your Django settings.

To a specific model, specify the MODEL_ID as below. The names of supported models can be found in the LiteLLM documentation.

WAGTAIL_VECTOR_INDEX = {
    "EMBEDDING_BACKENDS": {
        "default": {
            "CLASS": "wagtail_vector_index.ai_utils.backends.litellm.LiteLLMEmbeddingBackend",
            "CONFIG": {
                "MODEL_ID": "vertex_ai/textembedding-gecko",
            },
        },
    },
}

Using local models

To use local models with the LiteLLM backend, we recommend using the Ollama provider.