diff --git a/plans/llamacpp-cpu-backend-fix.md b/plans/llamacpp-cpu-backend-fix.md deleted file mode 100644 index f4b03fb4b..000000000 --- a/plans/llamacpp-cpu-backend-fix.md +++ /dev/null @@ -1,101 +0,0 @@ -# Fix Plan: llama.cpp CPU Backend Missing - -## Problem Summary - -The llama.cpp service fails to start with the error: -``` -llama_model_load: error loading model: make_cpu_buft_list: no CPU backend found -``` - -## Root Cause Analysis - -### Initial Hypothesis (INCORRECT) -Initially suspected that CPU backend files were missing from the installation. - -### Actual Root Cause (CONFIRMED) -The CPU backend files **ARE present** in `/opt/llamacpp/bin/`, but they fail to load due to a **missing dependency**: - -``` -$ ldd /opt/llamacpp/bin/libggml-cpu-haswell.so - libgomp.so.1 => not found <-- MISSING! -``` - -**`libgomp.so.1`** (GNU OpenMP runtime library) is required by the CPU backend libraries but is not installed in the container. - -### Why RPC and Vulkan Backends Load Successfully -- `libggml-rpc.so` - No `libgomp` dependency ✅ -- `libggml-vulkan.so` - No `libgomp` dependency ✅ -- `libggml-cpu-*.so` - **Requires `libgomp.so.1`** ❌ - -## Solution - -### Fix: Install libgomp1 Package - -Add `libgomp1` to the dependencies in [`install/llamacpp-install.sh`](install/llamacpp-install.sh): - -```bash -msg_info "Installing Dependencies" -$STD apt-get install -y \ - curl \ - wget \ - ca-certificates \ - vulkan-tools \ - libvulkan1 \ - mesa-vulkan-drivers \ - pciutils \ - libgomp1 # <-- Add this package -msg_ok "Installed Dependencies" -``` - -### Why This Works -- `libgomp1` provides `libgomp.so.1` (GNU OpenMP runtime) -- CPU backends use OpenMP for parallel processing -- Without this library, the dynamic linker fails to load `libggml-cpu-*.so` - -## Verification Steps - -After installing `libgomp1`, verify the fix: - -```bash -# 1. Verify libgomp is installed -ldd /opt/llamacpp/bin/libggml-cpu-haswell.so | grep gomp -# Expected: libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x...) - -# 2. Verify CPU backend loads -/opt/llamacpp/bin/llama-server --version -# Expected output should include: -# load_backend: loaded RPC backend from ... -# load_backend: loaded Vulkan backend from ... -# load_backend: loaded CPU backend from .../libggml-cpu-*.so <-- This should now appear - -# 3. Test model loading -systemctl start llamacpp -journalctl -u llamacpp -f -# Should no longer show "no CPU backend found" error -``` - -## Implementation - -### File to Modify -[`install/llamacpp-install.sh`](install/llamacpp-install.sh) - Line 16-23 - -### Change Required -```diff - msg_info "Installing Dependencies" - $STD apt-get install -y \ - curl \ - wget \ - ca-certificates \ - vulkan-tools \ - libvulkan1 \ - mesa-vulkan-drivers \ - pciutils \ -+ libgomp1 - msg_ok "Installed Dependencies" -``` - -## References - -- [GitHub Issue #17491: CPU backend loader does not respect LD_LIBRARY_PATH](https://github.com/ggml-org/llama.cpp/issues/17491) -- [GitHub Issue #14691: make_cpu_buft_list: no CPU backend found](https://github.com/ggml-org/llama.cpp/issues/14691) -- [GNU OpenMP (libgomp) Documentation](https://gcc.gnu.org/onlinedocs/libgomp/) diff --git a/plans/llamacpp-server-plan.md b/plans/llamacpp-server-plan.md deleted file mode 100644 index 61a731a8f..000000000 --- a/plans/llamacpp-server-plan.md +++ /dev/null @@ -1,178 +0,0 @@ -# llama.cpp Server Script Plan for ProxmoxVE - -## Overview - -This plan outlines the creation of a ProxmoxVE LXC container script for deploying **llama.cpp server** with GPU passthrough and Vulkan backend support. The script will install the latest Vulkan build and configure a default model for immediate use. - -## Reference Links - -- **GitHub Discussion**: https://github.com/community-scripts/ProxmoxVE/discussions/2403 -- **llama.cpp Repository**: https://github.com/ggml-org/llama.cpp -- **llama.cpp Server Documentation**: https://github.com/ggml-org/llama.cpp/tree/master/tools/server -- **Vulkan Build Downloads**: https://github.com/ggml-org/llama.cpp/releases - -## Implementation Status - -✅ **COMPLETED** - All files have been created. - -## Files Created - -### 1. Container Script: `ct/llamacpp.sh` - -**Purpose**: Main script for creating the LXC container - -**Key Features**: -- Debian 13 base (recommended for latest Vulkan support) -- GPU passthrough enabled via `var_gpu="yes"` -- 4 CPU cores, 8GB RAM minimum (model requires ~10GB for Q8_0) -- 20GB disk (model + binaries + cache) -- Auto-detects GPU type (AMD/Intel/NVIDIA) during updates -- Downloads appropriate build (Vulkan or CUDA) - -### 2. Installation Script: `install/llamacpp-install.sh` - -**Purpose**: Install llama.cpp with Vulkan support inside the container - -**Key Features**: -- Auto-detects GPU vendor (AMD/Intel/NVIDIA) -- Downloads latest Vulkan build from GitHub releases -- Falls back to Vulkan if CUDA download fails -- Creates systemd service with default model -- Configures GPU permissions automatically -- Creates GPU passthrough documentation file - -### 3. JSON Metadata: `frontend/public/json/llamacpp.json` - -**Purpose**: Frontend metadata for the script catalog - -**Key Features**: -- Category: AI/ML (0) -- Port: 8080 -- Updateable: true -- Comprehensive notes for GPU passthrough - -### 4. Header File: `ct/headers/llamacpp` - -**Purpose**: ASCII art header for the script - -## Default Configuration - -| Setting | Value | -|---------|-------| -| Model | `unsloth/Qwen3.5-9B-GGUF:Q8_0` | -| Port | 8080 | -| Context Size | 8192 tokens | -| GPU Layers | -1 (all layers on GPU) | -| Host | 0.0.0.0 (all interfaces) | - -## GPU Passthrough Configuration - -### AMD GPU Configuration - -Add to `/etc/pve/lxc/.conf`: -```ini -dev0: /dev/kfd,gid=104 -dev1: /dev/dri/renderD128,gid=104 -lxc.cgroup2.devices.allow: c 226:0 rwm -lxc.cgroup2.devices.allow: c 226:128 rwm -``` - -### Intel GPU Configuration - -Add to `/etc/pve/lxc/.conf`: -```ini -dev0: /dev/dri/renderD128,gid=104 -lxc.cgroup2.devices.allow: c 226:128 rwm -``` - -### NVIDIA GPU Configuration - -For NVIDIA, use the nvidia-container-toolkit approach: -```ini -lxc.cgroup2.devices.allow: c 195:* rwm -lxc.cgroup2.devices.allow: c 509:* rwm -``` - -## API Endpoints - -The server provides OpenAI-compatible endpoints: - -| Endpoint | Description | -|----------|-------------| -| `POST /v1/chat/completions` | Chat completion | -| `POST /v1/completions` | Text completion | -| `POST /v1/embeddings` | Text embeddings | -| `GET /v1/models` | List available models | -| `GET /health` | Health check | -| `GET /` | Built-in Web UI | - -## Resource Requirements - -| Resource | Minimum | Recommended | -|----------|---------|-------------| -| CPU | 2 cores | 4+ cores | -| RAM | 8GB | 16GB+ | -| Disk | 20GB | 50GB+ | -| GPU | Vulkan 1.1+ | Vulkan 1.3+ | - -## Testing Checklist - -- [ ] Container creation succeeds -- [ ] GPU passthrough is detected (`vulkaninfo` works) -- [ ] llama.cpp server starts successfully -- [ ] Model downloads automatically from HuggingFace -- [ ] API responds on port 8080 -- [ ] Chat completion endpoint works -- [ ] Update script functions correctly -- [ ] Service restarts after reboot - -## Usage Examples - -### Chat Completion (curl) - -```bash -curl http://localhost:8080/v1/chat/completions \ - -H "Content-Type: application/json" \ - -d '{ - "messages": [{"role": "user", "content": "Hello!"}], - "temperature": 0.7, - "max_tokens": 100 - }' -``` - -### Change Model - -```bash -# Edit the service file -nano /etc/systemd/system/llamacpp.service - -# Change the -hf parameter -# Example: -hf TheBloke/Llama-2-7B-GGUF:Q4_K_M - -# Reload and restart -systemctl daemon-reload -systemctl restart llamacpp -``` - -### Check Logs - -```bash -journalctl -u llamacpp -f -``` - -## Files Summary - -| File | Purpose | -|------|---------| -| `ct/llamacpp.sh` | Container creation script | -| `install/llamacpp-install.sh` | Installation script | -| `frontend/public/json/llamacpp.json` | Frontend metadata | -| `ct/headers/llamacpp` | ASCII art header | -| `plans/llamacpp-server-plan.md` | This documentation | - -## Next Steps - -1. Test the script on a Proxmox VE 8.x host with GPU passthrough -2. Verify GPU detection works for AMD, Intel, and NVIDIA -3. Test model download and inference -4. Verify update mechanism works correctly diff --git a/plans/wakapi-script-plan.md b/plans/wakapi-script-plan.md deleted file mode 100644 index a12bfe690..000000000 --- a/plans/wakapi-script-plan.md +++ /dev/null @@ -1,167 +0,0 @@ -# Wakapi ProxmoxVE Script Plan - -## Overview - -This plan outlines the creation of a ProxmoxVE community script for deploying **Wakapi** - a minimalist, self-hosted WakaTime-compatible backend for coding statistics. - -- **Source**: https://github.com/muety/wakapi -- **Website**: https://wakapi.dev -- **License**: MIT -- **Language**: Go -- **Default Port**: 3000 - -## Architecture - -```mermaid -flowchart TD - A[User runs ct/wakapi.sh] --> B[build.func processes request] - B --> C[Container created with Debian 13] - C --> D[install/wakapi-install.sh executed] - D --> E[Download Wakapi binary from GitHub] - E --> F[Create config.yml with SQLite] - F --> G[Create systemd service] - G --> H[Wakapi running on port 3000] -``` - -## Files to Create - -### 1. ct/wakapi.sh - -Main container creation script that: -- Defines container resources: 2 CPU, 2GB RAM, 8GB disk -- Uses Debian 13 as base OS -- Implements update_script function for version updates -- Sources build.func from the repository - -**Key Configuration:** -```bash -APP="Wakapi" -var_tags="productivity;analytics" -var_cpu="2" -var_ram="2048" -var_disk="8" -var_os="debian" -var_version="13" -var_unprivileged="1" -``` - -### 2. install/wakapi-install.sh - -Installation script that: -- Downloads the latest Wakapi binary from GitHub releases -- Creates necessary directories: /opt/wakapi -- Generates configuration file with SQLite backend -- Creates systemd service for automatic startup -- Generates a secure password salt - -**Installation Steps:** -1. Install dependencies: ca-certificates -2. Download Wakapi binary using fetch_and_deploy_gh_release -3. Create /opt/wakapi directory -4. Generate config.yml with: - - SQLite database - - Random password salt - - Proper security settings -5. Create systemd service -6. Enable and start service - -### 3. ct/headers/wakapi - -ASCII art header for the script output. - -### 4. frontend/public/json/wakapi.json - -Metadata file containing: -- Application name and slug -- Categories: productivity, analytics -- Installation methods -- Default credentials info -- Documentation links - -## Configuration Details - -### Wakapi Configuration (config.yml) - -```yaml -env: production -server: - port: 3000 - listen_ipv4: 0.0.0.0 - public_url: http://localhost:3000 - -security: - password_salt: - insecure_cookies: true - allow_signup: true - -db: - dialect: sqlite3 - name: /opt/wakapi/data/wakapi.db -``` - -### Systemd Service - -```ini -[Unit] -Description=Wakapi - Coding Statistics -After=network.target - -[Service] -Type=simple -User=root -WorkingDirectory=/opt/wakapi -ExecStart=/opt/wakapi/wakapi -config /opt/wakapi/config.yml -Restart=on-failure -RestartSec=5 - -[Install] -WantedBy=multi-user.target -``` - -## Update Strategy - -The update_script function will: -1. Check current version against GitHub releases -2. Stop the Wakapi service -3. Backup the config.yml and data directory -4. Download new binary -5. Restore configuration -6. Restart service - -## Resource Requirements - -| Resource | Value | Notes | -|----------|-------|-------| -| CPU | 2 cores | Sufficient for personal use | -| RAM | 2GB | Adequate for SQLite backend | -| Disk | 8GB | Stores binary, config, and database | -| OS | Debian 13 | Latest stable | - -## Post-Installation - -After installation, users need to: -1. Access Wakapi at http://IP:3000 -2. Create an account (first user becomes admin) -3. Configure WakaTime client with: - - API URL: http://IP:3000/api - - API Key: From user settings - -## Notes for Implementation - -1. **Binary Selection**: Wakapi provides pre-built binaries for Linux amd64/arm64. Use the appropriate binary based on architecture. - -2. **Password Salt**: Generate a random 32-character salt for password hashing. - -3. **Data Persistence**: The SQLite database is stored in /opt/wakapi/data/ which persists across updates. - -4. **No Default Credentials**: Wakapi requires user registration on first access - no default username/password. - -5. **Categories**: Use category IDs from the project - likely productivity and analytics related. - -## Implementation Checklist - -- [ ] Create ct/wakapi.sh with update_script function -- [ ] Create install/wakapi-install.sh with binary download and config -- [ ] Create ct/headers/wakapi ASCII art -- [ ] Create frontend/public/json/wakapi.json metadata -- [ ] Test the scripts in a Proxmox environment