From 0f6ebc19e97528be5e91ab6d85452eaf90d56f49 Mon Sep 17 00:00:00 2001 From: Classic298 <27028174+Classic298@users.noreply.github.com> Date: Tue, 21 Apr 2026 18:42:15 +0200 Subject: [PATCH] Claude/fix docs vtb qt (#3) --- .../extensibility/plugin/functions/filter.mdx | 12 ++++-- docs/reference/api-endpoints.md | 10 ++++- docs/troubleshooting/performance.md | 42 ++++++++++++++++--- 3 files changed, 53 insertions(+), 11 deletions(-) diff --git a/docs/features/extensibility/plugin/functions/filter.mdx b/docs/features/extensibility/plugin/functions/filter.mdx index 2485685b..9f68fb18 100644 --- a/docs/features/extensibility/plugin/functions/filter.mdx +++ b/docs/features/extensibility/plugin/functions/filter.mdx @@ -367,11 +367,17 @@ When using Open WebUI's API endpoints directly (e.g., via `curl` or external app |----------|--------------|-------------------| | `inlet()` | ✅ Always called | ✅ Always called | | `stream()` | ✅ Called during streaming | ✅ Called during streaming | -| `outlet()` | ✅ Called after response | ⚠️ Only if `chat_id` and `id` are present (see below) | +| `outlet()` | ✅ Called after response | ⚠️ Non-streaming only, and only if `chat_id` and `id` are present (see below) | | `__event_emitter__` | ✅ Shows UI feedback | ⚠️ Runs but no UI to display | +:::warning Streaming API Requests Do Not Invoke outlet() Yet +Inline `outlet()` execution during `/api/chat/completions` is currently only wired up for **non-streaming** direct API requests. For **streaming** (`"stream": true`) API requests, `outlet()` is not yet invoked by the backend — this is a known limitation that is being worked on. WebUI requests continue to run `outlet()` for both streaming and non-streaming responses as before. + +If you depend on `outlet()` for observability, logging, or post-processing of API traffic, either issue non-streaming requests or route the traffic through the WebUI until streaming support lands. +::: + :::warning Outlet Requires chat_id and message id -`outlet()` runs inline as part of `/api/chat/completions` processing. However, the outlet handler **requires both `chat_id` and `id` (message ID) to be present** in the request body. The WebUI always sends these, but pure API callers using the standard OpenAI-compatible request shape typically omit them — in which case **outlet is silently skipped**. +For non-streaming direct API requests, `outlet()` runs inline as part of `/api/chat/completions` processing. However, the outlet handler **requires both `chat_id` and `id` (message ID) to be present** in the request body. The WebUI always sends these, but pure API callers using the standard OpenAI-compatible request shape typically omit them — in which case **outlet is silently skipped**. If your filters rely on `outlet()` for API requests (logging, observability, post-processing, etc.), you need to either: 1. Include `chat_id` and `id` in your API request body, **or** @@ -1007,7 +1013,7 @@ The `outlet` function is like a **proofreader**: tidy up the AI's response (or m - **Quality scoring** - Run automated quality checks on model outputs :::info Outlet and API Requests -`outlet()` is called for direct API requests to `/api/chat/completions`, but only when `chat_id` and `id` (message ID) are present in the request body. See [Enabling Outlet for Pure API Callers](#enabling-outlet-for-pure-api-callers) for details. +`outlet()` is called for **non-streaming** direct API requests to `/api/chat/completions` when `chat_id` and `id` (message ID) are present in the request body. Streaming API requests do not yet invoke `outlet()` — support is in progress. See [Enabling Outlet for Pure API Callers](#enabling-outlet-for-pure-api-callers) for details. ::: 💡 **Example Use Case**: Strip out sensitive API responses you don't want the user to see: diff --git a/docs/reference/api-endpoints.md b/docs/reference/api-endpoints.md index fe8bdc06..f8ccbc44 100644 --- a/docs/reference/api-endpoints.md +++ b/docs/reference/api-endpoints.md @@ -197,16 +197,22 @@ Open WebUI accepts both **API keys** (prefixed with `sk-`) and **JWT tokens** fo |----------------|--------------|-------------------| | `inlet()` | ✅ Runs | ✅ Runs | | `stream()` | ✅ Runs | ✅ Runs | -| `outlet()` | ✅ Runs | ✅ Runs | +| `outlet()` | ✅ Runs | ⚠️ Non-streaming only (see note) | The `inlet()` function always executes, making it ideal for: - **Rate limiting** - Track and limit requests per user - **Request logging** - Log all API usage for monitoring - **Input validation** - Reject invalid requests before they reach the model +:::warning Streaming Requests +Inline `outlet()` execution during `/api/chat/completions` is currently only wired up for **non-streaming** direct API requests. For **streaming** (`"stream": true`) API requests, `outlet()` is not yet invoked by the backend — this is a known limitation that is being worked on. WebUI requests continue to run `outlet()` for both streaming and non-streaming responses as before. + +If you rely on `outlet()` for observability, logging, or post-processing of streamed API responses, use non-streaming requests or consume the response via the WebUI until streaming support lands. +::: + #### Legacy Endpoint: `/api/chat/completed` -`outlet()` now runs inline during `/api/chat/completions` for both WebUI and direct API requests. +`outlet()` now runs inline during `/api/chat/completions` for WebUI requests and for non-streaming direct API requests (see caveat above). `POST /api/chat/completed` is retained for backward compatibility with older clients that still call it as a second step: diff --git a/docs/troubleshooting/performance.md b/docs/troubleshooting/performance.md index 79ad4375..74dc0b68 100644 --- a/docs/troubleshooting/performance.md +++ b/docs/troubleshooting/performance.md @@ -292,8 +292,27 @@ If you're using **SQLite** (the default) in a cloud environment, you may be trad Cloud storage (Azure Disks, AWS EBS, GCP Persistent Disks) often has significantly higher latency and lower IOPS than local NVMe/SSD storage—especially on lower-tier storage classes. -:::warning Warning: Performance Risk with Network File Systems -Using Network-attached File Systems like **NFS, SMB, or Azure Files** for your database storage (especially for SQLite) **may** introduce severe latency into the file locking and synchronous write operations that SQLite relies on. +:::danger SQLite on NFS / SMB / Azure Files Is Not Supported — by SQLite Itself +This restriction does **not** come from Open WebUI — it comes from **SQLite upstream**. The SQLite project [officially states](https://www.sqlite.org/faq.html#q5) that SQLite databases on network filesystems (NFS, SMB/CIFS, and similar) are **not supported**: file locking over those protocols is unreliable, and concurrent writers **can corrupt the database**. The SQLite documentation explicitly warns against it, and Open WebUI inherits that constraint because it uses SQLite. + +As a consequence, for Open WebUI the **only supported storage configurations are**: + +- **PostgreSQL** — recommended for any multi-user deployment and required for anything not on a directly-attached local SSD. This sidesteps the SQLite-on-network-storage problem entirely. **Or** +- **SQLite on a directly-attached SSD / NVMe** — single-user / small deployments only. Must be a **local** disk on the host; SQLite upstream's guidance applies regardless of the application. + +**Not supported** for SQLite (per SQLite's own documentation, not an Open WebUI policy): **NFS, SMB/CIFS, Azure Files, GlusterFS, CephFS, object-storage-backed FUSE mounts, network PVCs, any remote or low-IOPS storage.** This includes Docker bind mounts and Kubernetes PersistentVolumeClaims backed by those filesystems. Beyond the performance symptoms below, you are risking **database corruption** — again, per SQLite, not us. + +If your storage is anything other than a local SSD/NVMe, **use PostgreSQL**. + +Typical symptoms after upgrading to releases that use the async SQLite driver: + +- `/api/config` takes **10–20+ seconds** on every request +- `/api/v1/chats/?page=1` and other list endpoints stall for **minutes** under load +- OIDC / SSO callbacks hang or "spin" when redirecting back to Open WebUI +- Large (multi-second) gaps in DEBUG logs between `aiosqlite` and `httpcore` lines +- `PRAGMA journal_mode=WAL` starts but never completes in logs + +Older synchronous SQLAlchemy releases (≤ 0.8.12) serialized contention in-process, which masked slow storage. The async driver opens connections across threads and hammers the filesystem, so network-attached storage degradation becomes immediately visible. ::: SQLite is particularly sensitive to disk performance because it performs synchronous writes. Moving from local SSDs to a network share can increase latency by 10x or more per operation. @@ -302,10 +321,21 @@ SQLite is particularly sensitive to disk performance because it performs synchro - Performance is acceptable with a single user but degrades rapidly as concurrency increases. - High "I/O Wait" on the server despite low CPU usage. -**Solutions:** -1. **Use high-performance Block Storage:** - - Ensure you are using SSD-backed **Block Storage** classes (e.g., `Premium_LRS` on Azure Disks, `gp3` on AWS EBS, `pd-ssd` on GCP). Avoid "File" based storage classes (like `azurefile-csi`) for database workloads. -2. **Use PostgreSQL instead:** For any medium to large production deployment, **Postgres is mandatory**. SQLite is generally not recommended at scale in cloud environments due to the inherent latency of network-attached storage and the compounding effect of file locking over the network. +**Solutions (in order of robustness):** + +1. **Best — migrate to PostgreSQL.** This is the recommended fix for any deployment that is not strictly single-user on a local disk, and it is required for any deployment on remote / network / low-IOPS storage. Set: + ```bash + DATABASE_URL=postgresql+asyncpg://user:password@host:5432/webui + ``` + PostgreSQL removes the fsync-per-connection pathology entirely because the database process owns its own storage, and it is the only supported option for multi-user workloads. +2. **Acceptable — move `webui.db` onto directly-attached local SSD/NVMe.** Only appropriate for single-user or very small deployments. Bind-mount a directory on the host's **local** SSD/NVMe into `/app/backend/data`. Do **not** use NFS, SMB, Azure Files, or any network-backed storage class — not even "high-performance" network block storage. SQLite was not designed for network filesystems and will always be slow on them. +3. **Temporary workaround only — keep SQLite on NFS with reduced concurrency.** If you cannot immediately move storage or switch databases, set: + ```bash + DATABASE_POOL_SIZE=1 + DATABASE_SQLITE_PRAGMA_BUSY_TIMEOUT=30000 + ``` + `DATABASE_POOL_SIZE=1` forces a single serialized async connection, trading concurrency for stability. `DATABASE_SQLITE_PRAGMA_BUSY_TIMEOUT=30000` gives SQLite 30 seconds to acquire locks, which NFS can take much longer to grant than local disk. This is **not a supported long-term configuration** — expect degraded throughput, intermittent stalls, and potential corruption. Plan to migrate to PostgreSQL or local SSD as soon as possible. A warm pool may briefly appear fine after restart, but the problem returns under load. +4. **Cloud block storage:** When using cloud block storage for the Open WebUI data volume (for PostgreSQL or the application itself), use SSD-backed **Block Storage** classes (e.g., `Premium_LRS` on Azure Disks, `gp3` on AWS EBS, `pd-ssd` on GCP). Avoid "File" based storage classes (like `azurefile-csi`) for any database workload — including SQLite. ### Other Cloud-Specific Considerations