> For the complete documentation index, see [llms.txt](https://guide.tiledesk.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://guide.tiledesk.com/data-sovereign-installations-local-data-local-ai.md).

# Data-sovereign installations: local data, local AI

Install the full platform services stack, including LLMs, embedding models and vector stores, on your own infrastructure

## Overview

Tiledesk supports data-sovereign installation: you can install the full platform services stack, including LLMs, embedding models and vector stores, on your own infrastructure. No data, whether conversations, prompts, documents or vectors, is sent to external services.

This article describes:

* the supported installation environments
* how LLM and embedding models are served locally
* which platform modules use local models
* how to configure models and vector stores at platform, project and knowledge base level
* how to combine multiple knowledge bases with the Agent Designer

> **Terminology.** In Tiledesk, tenants are called **projects**. The two terms are used interchangeably in this article.

### Supported installation environments

Data-sovereign installation is available on the major public clouds and in the customer's private data center. In every case, the platform and the models run on resources controlled by the customer.

| Environment                 | Type         | Notes                                                              |
| --------------------------- | ------------ | ------------------------------------------------------------------ |
| Amazon Web Services (AWS)   | Public cloud | Installed in the account and region chosen by the customer         |
| Google Cloud Platform (GCP) | Public cloud | Installed in the project and region chosen by the customer         |
| Microsoft Azure             | Public cloud | Installed in the subscription and region chosen by the customer    |
| Private data center         | On-premise   | Installed on the customer's physical or virtualized infrastructure |

The choice of environment does not change the capabilities described in this article: local models, per-project configuration and per-knowledge-base vector stores are available on every kind of installation.

### Local LLM and embedding models

In a data-sovereign installation, both Large Language Models and embedding models are served locally. Tiledesk integrates with two inference runtimes:

| Runtime | Typical use                                                               |
| ------- | ------------------------------------------------------------------------- |
| vLLM    | Production environments with high throughput and many concurrent requests |
| Ollama  | Lightweight installations, development or test environments               |

The customer chooses which models to run on the runtime, based on language, size, license and available hardware. The embedding model is local too: document vectorization happens entirely within the customer's infrastructure.

### Modules that use local models <a href="#modules-that-use-local-models" id="modules-that-use-local-models"></a>

Every AI-enabled module of the platform can use the models installed on vLLM or Ollama.

| Module                          | Function                                                    | Models used                                                            |
| ------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------- |
| RAG                             | Generates answers from the project's knowledge bases        | LLM (generation), embedding (retrieval)                                |
| OCR                             | Extracts text from images and scanned documents             | It uses a local running Tesseract or Models configured by the customer |
| Document ingestion              | Acquires, chunks and indexes documents into knowledge bases | Paddle for non-planar document images                                  |
| Agents and conversational flows | Orchestrate dialogue, tools and knowledge bases             | LLM                                                                    |

The entire chain, from document upload to generated answer, therefore runs with no dependency on external services.

### Per-project model configuration

Tiledesk is a multi-tenant platform. Model configuration follows a two-level inheritance model:

1. **Platform level.** The installation declares the main models (LLM and embedding). These act as defaults for all projects.
2. **Project level.** Each project can redeclare its models, replacing the platform defaults with those best suited to its goals. A project that redeclares nothing inherits the defaults.

Examples of per-project configuration:

* a high-volume customer care project uses a compact, low-latency LLM;
* a document analysis project uses a larger LLM and a specialized embedding model;
* a multilingual project uses an embedding model optimized for the languages it handles.

Projects with different configurations coexist in the same installation.

### Supported vector stores

The vector store, where the vectors of indexed documents reside, is local and interchangeable. The following engines are supported:

| Vector store | Persistence | Recommended use                          |
| ------------ | ----------- | ---------------------------------------- |
| Qdrant       | Persistent  | Default vector store for most use cases  |
| Milvus       | Persistent  | Alternative to Qdrant, for big data sets |
| Redis Vector | Persistent  | Scenarios that require very low latency  |

### Per-knowledge-base vector store configuration

As with models, the vector store follows an inheritance model. The default is declared at platform level, and each knowledge base can replace it with its own. Vector data residency is therefore defined per project and, within each project, per knowledge base.

<figure><img src="https://1328774413-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVguk6y5OMn3pAaHZSiwU%2Fuploads%2FslsKW6nZtVVgLqlE3qgC%2Fimage.png?alt=media&amp;token=641d1aad-404e-4a4f-aca0-39490f9ef016" alt=""><figcaption></figcaption></figure>

In the example, Project A redeclares its models and uses three knowledge bases on two different vector stores; Project B inherits all platform defaults. Both projects live in the same installation.

Configuration levels:

1. **Platform.** The installation declares the default vector store, for example Qdrant.
2. **Project.** Each project can define N isolated knowledge bases (RAG).
3. **Knowledge base.** Each knowledge base can declare its own vector store. If it does not, it inherits the platform default.

Examples:

* a small knowledge base that requires minimal response times uses Redis Vector
* the remaining knowledge bases inherit Qdrant

### Combining knowledge bases with the Agent Designer

A project's knowledge bases can be freely combined using the **Agent Designer**. For each agent you can define which knowledge bases to query and in what order.

Example of a fallback chain:

1. The agent searches for the answer in KB 1.
2. If KB 1 does not return an adequate answer, the agent searches KB 2.
3. If KB 2 does not return an adequate answer either, the agent searches KB 3.

Since each knowledge base can use a different vector store, a single chain can query data residing on different engines, for example redis and Qdrant Vector.

### Configuration hierarchy summary

| Level            | LLM and embedding models                | Vector store                                  |
| ---------------- | --------------------------------------- | --------------------------------------------- |
| Platform         | Declared as installation defaults       | Declared as installation default              |
| Project (tenant) | Inherited, or redeclared by the project | No declaration: configured per knowledge base |
| Knowledge base   | Those of the project                    | Inherited, or declared by the knowledge base  |

The main models are therefore defined once at platform level, while each project can redeclare and combine them according to the needs of each individual customer.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://guide.tiledesk.com/data-sovereign-installations-local-data-local-ai.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
