mirror of
https://github.com/open-webui/docs.git
synced 2026-07-21 18:15:36 -04:00
refac
This commit is contained in:
Timothy Jaeryang Baek
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"label": "🖥️ Open WebUI Computer",
|
||||
"label": "🖥️ Open WebUI Computer (cptr)",
|
||||
"position": 36,
|
||||
"collapsible": true,
|
||||
"collapsed": true
|
||||
|
||||
@@ -9,4 +9,5 @@ Open WebUI is the core AI interface. The ecosystem is the set of separate Open W
|
||||
|
||||
## Projects
|
||||
|
||||
- [**Knowledge Base Sync (oikb)**](/ecosystem/knowledge-base-sync): mirror folders, repos, buckets, wikis, and other sources into Open WebUI Knowledge Bases with incremental sync.
|
||||
- [**Open WebUI Computer**](/ecosystem/computer): your computer in a browser tab, with files, terminal, git, editor, and optional AI. It connects to Open WebUI through an OpenAI-compatible gateway.
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"label": "🔄 Knowledge Base Sync",
|
||||
"label": "🔄 Knowledge Base Sync (oikb)",
|
||||
"position": 34,
|
||||
"collapsible": true,
|
||||
"collapsed": true
|
||||
+3
-3
@@ -5,7 +5,7 @@ title: "Daemon, Webhooks & Deployment"
|
||||
|
||||
# Running the daemon
|
||||
|
||||
The CLI on the [main page](/features/knowledge-base-sync) covers one-off and watched syncs. For anything unattended, scheduled or driven by external events, run oikb as a long-lived **daemon**. It reads your [`.oikb.yaml`](#the-oikbyaml-config-file), syncs each source on its own schedule and exposes a small HTTP API for health checks, metrics, history and on-demand triggers.
|
||||
The CLI on the [main page](/ecosystem/knowledge-base-sync) covers one-off and watched syncs. For anything unattended, scheduled or driven by external events, run oikb as a long-lived **daemon**. It reads your [`.oikb.yaml`](#the-oikbyaml-config-file), syncs each source on its own schedule and exposes a small HTTP API for health checks, metrics, history and on-demand triggers.
|
||||
|
||||
```bash
|
||||
oikb daemon
|
||||
@@ -331,12 +331,12 @@ The scheduler and per-KB locks live in one process, so the daemon is meant to ru
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
For push-based, build-time syncing instead of a long-running daemon, run the image as a one-shot step. See [One-shot sync in CI](/features/knowledge-base-sync#one-shot-sync-in-ci) on the main page.
|
||||
For push-based, build-time syncing instead of a long-running daemon, run the image as a one-shot step. See [One-shot sync in CI](/ecosystem/knowledge-base-sync#one-shot-sync-in-ci) on the main page.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- **[Knowledge Base Sync (oikb)](/features/knowledge-base-sync)**: installation, sources, connectors, filtering and the CLI.
|
||||
- **[Knowledge Base Sync (oikb)](/ecosystem/knowledge-base-sync)**: installation, sources, connectors, filtering and the CLI.
|
||||
- **[Knowledge](/features/workspace/knowledge)**: the Knowledge Base feature this keeps in sync.
|
||||
- **[OpenAPI / MCP tool servers](/features/extensibility/mcp)**: connecting external tool servers to Open WebUI.
|
||||
+13
-13
@@ -1,7 +1,7 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
title: "Knowledge Base Sync (oikb)"
|
||||
slug: /features/knowledge-base-sync
|
||||
slug: /ecosystem/knowledge-base-sync
|
||||
---
|
||||
|
||||
import ThemedImage from '@theme/ThemedImage';
|
||||
@@ -24,7 +24,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
|
||||
Unlike a one-time upload, which is stale the moment the source changes, oikb does **incremental** sync: it wires a Knowledge Base to a living source once, then keeps it fresh on a schedule, on every push or on demand. A docs repo, a wiki, a storage bucket, all stay current without anyone babysitting them.
|
||||
|
||||
It is a separate program (a command-line tool plus an optional long-running [daemon](/features/knowledge-base-sync/daemon)), not part of the Open WebUI server, but it is built for Open WebUI and talks to it over the normal REST API.
|
||||
It is a separate program (a command-line tool plus an optional long-running [daemon](/ecosystem/knowledge-base-sync/daemon)), not part of the Open WebUI server, but it is built for Open WebUI and talks to it over the normal REST API.
|
||||
|
||||
:::info Requires Open WebUI 0.9.6+
|
||||
oikb drives the incremental sync endpoints (`/sync/diff` and `/sync/cleanup`) that landed in **v0.9.6**. Against an older server there is nothing for it to call. The server side of these endpoints is documented under [Knowledge → Syncing a local directory](/features/workspace/knowledge#syncing-a-local-directory).
|
||||
@@ -48,11 +48,11 @@ Point it at a local folder, a Git repo, a Confluence or Notion space, an S3 buck
|
||||
|
||||
### Stays fresh on its own
|
||||
|
||||
Sync once by hand, or hand it to the [daemon](/features/knowledge-base-sync/daemon) to run on an interval, a cron or the instant someone pushes. A Knowledge Base wired to a living source stops quietly drifting out of date.
|
||||
Sync once by hand, or hand it to the [daemon](/ecosystem/knowledge-base-sync/daemon) to run on an interval, a cron or the instant someone pushes. A Knowledge Base wired to a living source stops quietly drifting out of date.
|
||||
|
||||
### The model can drive it
|
||||
|
||||
The daemon doubles as an [OpenAPI tool server](/features/knowledge-base-sync/daemon#let-the-model-trigger-syncs), so a model can trigger a re-sync and report back mid-conversation, without anyone touching the command line.
|
||||
The daemon doubles as an [OpenAPI tool server](/ecosystem/knowledge-base-sync/daemon#let-the-model-trigger-syncs), so a model can trigger a re-sync and report back mid-conversation, without anyone touching the command line.
|
||||
|
||||
---
|
||||
|
||||
@@ -90,7 +90,7 @@ pip install oikb[all] # every optional connector at once
|
||||
|
||||
The full set of extras is `s3`, `gcs`, `azure`, `dropbox`, `r2`, `gdrive`, `gmail`, `gsites`, `web`, `oracle`, `sharepoint-cert` and `all`. Connectors not listed here (GitHub, Confluence, Notion, Jira, Slack and most others) need no extra.
|
||||
|
||||
For production, a Docker image is published at `ghcr.io/open-webui/oikb`. See [Running the daemon](/features/knowledge-base-sync/daemon#deployment).
|
||||
For production, a Docker image is published at `ghcr.io/open-webui/oikb`. See [Running the daemon](/ecosystem/knowledge-base-sync/daemon#deployment).
|
||||
|
||||
---
|
||||
|
||||
@@ -164,7 +164,7 @@ The matching server-side behaviour and the same two endpoints exposed for your o
|
||||
|
||||
## Sources and connectors
|
||||
|
||||
A **source** is either a local path or a `scheme:target` string. oikb resolves the scheme to a connector, pulls the files and syncs them into the Knowledge Base. Credentials are never written into the source string or `.oikb.yaml`; each connector reads them from **environment variables**, so secrets stay out of your config (pair with [`${VAR}` interpolation](/features/knowledge-base-sync/daemon#the-oikbyaml-config-file) to keep config files committable).
|
||||
A **source** is either a local path or a `scheme:target` string. oikb resolves the scheme to a connector, pulls the files and syncs them into the Knowledge Base. Credentials are never written into the source string or `.oikb.yaml`; each connector reads them from **environment variables**, so secrets stay out of your config (pair with [`${VAR}` interpolation](/ecosystem/knowledge-base-sync/daemon#the-oikbyaml-config-file) to keep config files committable).
|
||||
|
||||
In the table below, `[brackets]` mark optional parts of the source string.
|
||||
|
||||
@@ -225,7 +225,7 @@ A few connector notes worth knowing:
|
||||
|
||||
## Filtering what gets synced
|
||||
|
||||
Narrow a source with include/exclude globs and a size cap. These live under `filter:` in [`.oikb.yaml`](/features/knowledge-base-sync/daemon#the-oikbyaml-config-file):
|
||||
Narrow a source with include/exclude globs and a size cap. These live under `filter:` in [`.oikb.yaml`](/ecosystem/knowledge-base-sync/daemon#the-oikbyaml-config-file):
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
@@ -270,13 +270,13 @@ For a local directory you are actively editing, `watch` re-syncs the moment a fi
|
||||
oikb watch ./docs --kb-id your-kb-id
|
||||
```
|
||||
|
||||
It uses filesystem events (not polling), debounced one second (tune with `--debounce`), and runs until you stop it with Ctrl+C. Good for a live notes folder; for anything remote or unattended, use the [daemon](/features/knowledge-base-sync/daemon) instead.
|
||||
It uses filesystem events (not polling), debounced one second (tune with `--debounce`), and runs until you stop it with Ctrl+C. Good for a live notes folder; for anything remote or unattended, use the [daemon](/ecosystem/knowledge-base-sync/daemon) instead.
|
||||
|
||||
---
|
||||
|
||||
## One-shot sync in CI
|
||||
|
||||
The Docker image runs as a one-shot command, which makes it a drop-in [GitHub Actions](/features/knowledge-base-sync/daemon#deployment) step to push docs into a Knowledge Base on every merge:
|
||||
The Docker image runs as a one-shot command, which makes it a drop-in [GitHub Actions](/ecosystem/knowledge-base-sync/daemon#deployment) step to push docs into a Knowledge Base on every merge:
|
||||
|
||||
```yaml
|
||||
- name: Sync docs to Open WebUI
|
||||
@@ -294,7 +294,7 @@ The Docker image runs as a one-shot command, which makes it a drop-in [GitHub Ac
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `oikb sync <source>` | Incremental sync from a source to a KB (`--kb-id`). Omit `<source>` to sync every entry in [`.oikb.yaml`](/features/knowledge-base-sync/daemon#the-oikbyaml-config-file). |
|
||||
| `oikb sync <source>` | Incremental sync from a source to a KB (`--kb-id`). Omit `<source>` to sync every entry in [`.oikb.yaml`](/ecosystem/knowledge-base-sync/daemon#the-oikbyaml-config-file). |
|
||||
| `oikb sync --dry-run` | Preview added/modified/deleted without uploading. |
|
||||
| `oikb sync --concurrency N` | Upload N files in parallel (default 1, sequential). |
|
||||
| `oikb sync --max-file-size 50mb` | Skip files above a size. |
|
||||
@@ -303,7 +303,7 @@ The Docker image runs as a one-shot command, which makes it a drop-in [GitHub Ac
|
||||
| `oikb watch <dir> --kb-id ID` | Auto-sync a local directory on change (`--debounce`). |
|
||||
| `oikb init` | Interactive wizard that writes a `.oikb.yaml`. |
|
||||
| `oikb validate` | Check `.oikb.yaml` syntax. Add `--deep` to also ping Open WebUI, verify the API key and confirm each KB exists. |
|
||||
| `oikb daemon` | Run the scheduled [daemon](/features/knowledge-base-sync/daemon) with an HTTP API. |
|
||||
| `oikb daemon` | Run the scheduled [daemon](/ecosystem/knowledge-base-sync/daemon) with an HTTP API. |
|
||||
| `oikb ls --kb-id ID` | List files in a KB. |
|
||||
| `oikb status --kb-id ID` | Show a KB's name, file count and total size. |
|
||||
| `oikb history` | View sync history (`--json`, `--errors`, `--kb-id`, `--clear --days N`). |
|
||||
@@ -350,7 +350,7 @@ The incremental sync endpoints landed in v0.9.6. Against an older server there i
|
||||
|
||||
### The daemon runs as one process
|
||||
|
||||
Scheduling and the per-KB locks live in a single process, so the [daemon](/features/knowledge-base-sync/daemon#deployment) is meant to run as one replica. To cover more sources, add entries to one daemon rather than running several copies.
|
||||
Scheduling and the per-KB locks live in a single process, so the [daemon](/ecosystem/knowledge-base-sync/daemon#deployment) is meant to run as one replica. To cover more sources, add entries to one daemon rather than running several copies.
|
||||
|
||||
### Indexing still happens server-side
|
||||
|
||||
@@ -374,6 +374,6 @@ oikb uploads fast, but Open WebUI extracts and embeds each new file asynchronous
|
||||
|
||||
## See also
|
||||
|
||||
- **[Running the daemon →](/features/knowledge-base-sync/daemon)**: scheduled sync, webhooks, the HTTP API, observability, deployment and letting a model trigger syncs.
|
||||
- **[Running the daemon →](/ecosystem/knowledge-base-sync/daemon)**: scheduled sync, webhooks, the HTTP API, observability, deployment and letting a model trigger syncs.
|
||||
- **[Knowledge](/features/workspace/knowledge)**: the Knowledge Base feature oikb feeds, including the server-side [sync endpoints](/features/workspace/knowledge#api-access).
|
||||
- **[OpenAPI / MCP tool servers](/features/extensibility/mcp)**: how the daemon plugs in as a tool the model can call.
|
||||
@@ -11,7 +11,7 @@ This repository provides reference OpenAPI Tool Server implementations making it
|
||||
By leveraging OpenAPI, we eliminate the need for a proprietary or unfamiliar communication protocol, ensuring you can quickly and confidently build or integrate servers. This means less time spent figuring out custom interfaces and more time building powerful tools that enhance your AI applications.
|
||||
|
||||
:::tip A first-party example
|
||||
The [oikb](/features/knowledge-base-sync) sync daemon is a ready-made OpenAPI tool server. Run it, add its URL under **Settings → Tools**, and a model can trigger Knowledge Base syncs and check their status from chat. See [Let the model trigger syncs](/features/knowledge-base-sync/daemon#let-the-model-trigger-syncs).
|
||||
The [oikb](/ecosystem/knowledge-base-sync) sync daemon is a ready-made OpenAPI tool server. Run it, add its URL under **Settings → Tools**, and a model can trigger Knowledge Base syncs and check their status from chat. See [Let the model trigger syncs](/ecosystem/knowledge-base-sync/daemon#let-the-model-trigger-syncs).
|
||||
:::
|
||||
|
||||
## ☝️ Why OpenAPI?
|
||||
|
||||
@@ -65,9 +65,8 @@ Upload files, build knowledge bases, and let the AI retrieve exactly the informa
|
||||
| 📑 **8 extraction engines** | Tika, Docling, Azure, Mistral OCR, Datalab Marker, MinerU, PaddleOCR, custom loaders |
|
||||
| 🤖 **Agentic retrieval** | Models search and read your documents autonomously |
|
||||
| 📄 **Full context mode** | Inject entire documents - no chunking, no guessing |
|
||||
| 🔄 **Knowledge Base sync** | Keep a KB current with a folder, Git repo, wiki, or storage bucket via [oikb](/features/knowledge-base-sync) |
|
||||
|
||||
[**Learn about Knowledge →**](/features/workspace/knowledge) · [**Knowledge Base Sync →**](/features/knowledge-base-sync)
|
||||
[**Learn about Knowledge →**](/features/workspace/knowledge)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ Attach specific knowledge bases to a model so it only searches what's relevant.
|
||||
| 🤖 **Agentic retrieval** | Models browse, search, and read your documents autonomously |
|
||||
| 📄 **Full context mode** | Inject entire documents with no chunking |
|
||||
| 🗂️ **Nested directories** | Organize files into subdirectories with drag-and-drop reordering |
|
||||
| 🔄 **Incremental directory sync** | Mirror a local folder into the KB, only new and modified files upload, deletions are removed, folder structure preserved. Mirror remote sources too (Git, Confluence, S3 and more) with [oikb](/features/knowledge-base-sync) |
|
||||
| 🔄 **Incremental directory sync** | Mirror a local folder into the KB, only new and modified files upload, deletions are removed, folder structure preserved. Mirror remote sources too (Git, Confluence, S3 and more) with [oikb](/ecosystem/knowledge-base-sync) |
|
||||
| 📦 **Export and API** | Back up knowledge bases as zip files, manage via REST API |
|
||||
|
||||
---
|
||||
@@ -252,7 +252,7 @@ Individual files can be renamed in place from the workspace via the file's item
|
||||
The **Add Content → Sync Directory** action mirrors a local folder into the knowledge base **incrementally**: the client hashes each local file (SHA-256), the server compares hashes and paths against what is already stored, and only **new**, **modified**, and **deleted** files are touched. Unmodified files (the typical majority) are left alone: no re-upload, no re-embedding. The local folder's subdirectory structure is mirrored in the KB; missing subdirectories are created, and subdirectories that no longer exist locally are removed.
|
||||
|
||||
:::tip Syncing a lot of files? Use oikb instead
|
||||
The in-app **Sync Directory** runs in your browser: it enumerates, hashes and uploads every file client-side. That is fine for a modest folder, but on a **large** set (thousands of files or a multi-gigabyte vault) it gets slow and gives little progress feedback, often sitting for a long time before it even reports the file count. For large or frequently-changing libraries we **strongly recommend** the official [**Knowledge Base Sync (oikb)**](/features/knowledge-base-sync) tool instead: it runs natively with progress bars, parallel uploads and retries, built for exactly this. Point it at the top of your folder and it syncs the entire tree (every subdirectory, recursively) into one Knowledge Base.
|
||||
The in-app **Sync Directory** runs in your browser: it enumerates, hashes and uploads every file client-side. That is fine for a modest folder, but on a **large** set (thousands of files or a multi-gigabyte vault) it gets slow and gives little progress feedback, often sitting for a long time before it even reports the file count. For large or frequently-changing libraries we **strongly recommend** the official [**Knowledge Base Sync (oikb)**](/ecosystem/knowledge-base-sync) tool instead: it runs natively with progress bars, parallel uploads and retries, built for exactly this. Point it at the top of your folder and it syncs the entire tree (every subdirectory, recursively) into one Knowledge Base.
|
||||
:::
|
||||
|
||||
Behavior to be aware of:
|
||||
@@ -265,7 +265,7 @@ Behavior to be aware of:
|
||||
For programmatic use, the same workflow is exposed as two endpoints under [API access](#api-access) below.
|
||||
|
||||
:::tip Sync remote sources, or on a schedule
|
||||
The in-app **Sync Directory** action handles a local folder on your machine. To mirror a **remote** source instead (a GitHub repo, a Confluence space, an S3 bucket and [dozens more](/features/knowledge-base-sync#sources-and-connectors)), or to keep a Knowledge Base current **automatically** on a schedule or on every push, use the official [**Knowledge Base Sync (oikb)**](/features/knowledge-base-sync) companion tool. It drives these same endpoints for you.
|
||||
The in-app **Sync Directory** action handles a local folder on your machine. To mirror a **remote** source instead (a GitHub repo, a Confluence space, an S3 bucket and [dozens more](/ecosystem/knowledge-base-sync#sources-and-connectors)), or to keep a Knowledge Base current **automatically** on a schedule or on every push, use the official [**Knowledge Base Sync (oikb)**](/ecosystem/knowledge-base-sync) companion tool. It drives these same endpoints for you.
|
||||
:::
|
||||
|
||||
### Exporting
|
||||
|
||||
Reference in New Issue
Block a user