Release 0.5.20 (#671)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

.github/workflows/e2e.yml

@@ -64,6 +64,15 @@ jobs:
         run: pnpm run pack-install
         working-directory: packages/create-llama
 
+      - name: Build and store server package
+        run: |
+          pnpm run build
+          wheel_file=$(ls dist/*.whl | head -n 1)
+          mkdir -p "${{ runner.temp }}"
+          cp "$wheel_file" "${{ runner.temp }}/"
+          echo "SERVER_PACKAGE_PATH=${{ runner.temp }}/$(basename "$wheel_file")" >> $GITHUB_ENV
+        working-directory: python/llama-index-server
+
       - name: Run Playwright tests for Python
         run: pnpm run e2e:python
         env:
@@ -74,6 +83,7 @@ jobs:
           TEMPLATE_TYPE: ${{ matrix.template-types }}
           PYTHONIOENCODING: utf-8
           PYTHONLEGACYWINDOWSSTDIO: utf-8
+          SERVER_PACKAGE_PATH: ${{ env.SERVER_PACKAGE_PATH }}
         working-directory: packages/create-llama
 
       - uses: actions/upload-artifact@v4

.github/workflows/release.yml

@@ -56,6 +56,8 @@ jobs:
         with:
           commit: Release ${{ steps.get-changeset-status.outputs.new-version }}
           title: Release ${{ steps.get-changeset-status.outputs.new-version }}
+          # bump versions
+          version: pnpm new-version
           # build package and call changeset publish
           publish: pnpm release
         env:

.prettierignore

@@ -7,6 +7,7 @@ build/
 .next/
 out/
 packages/server/server/
+packages/server/project/
 **/playwright-report/
 **/test-results/
 

CLAUDE.md

@@ -0,0 +1,201 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Repository Overview
+
+Create-llama is a monorepo containing CLI tools and server frameworks for building LlamaIndex-powered applications. The repository combines TypeScript/Node.js and Python components in a unified development environment.
+
+## Architecture
+
+### Monorepo Structure
+
+- **`packages/create-llama/`**: Main CLI tool for scaffolding LlamaIndex applications
+- **`packages/server/`**: TypeScript/Next.js server framework (`@llamaindex/server`)
+- **`python/llama-index-server/`**: Python/FastAPI server framework
+- **Root**: Workspace configuration and shared development tools
+
+### Key Technologies
+
+- **Package Manager**: pnpm with workspace configuration
+- **Build Tools**: bunchee (TypeScript), Next.js, hatchling (Python)
+- **Testing**: Playwright for e2e, pytest for Python
+- **Version Management**: changesets for TypeScript packages, manual for Python
+
+## Development Commands
+
+### Root Level (Monorepo)
+
+```bash
+pnpm dev          # Start all packages in development mode
+pnpm build        # Build all packages
+pnpm lint         # ESLint across TypeScript packages
+pnpm format       # Prettier formatting
+pnpm e2e          # Run end-to-end tests
+```
+
+### Create-llama Package
+
+```bash
+cd packages/create-llama
+npm run build     # Build CLI using bash script and ncc
+npm run dev       # Watch mode development
+npm run e2e       # Playwright tests for generated projects
+npm run clean     # Clean build artifacts and template caches
+```
+
+### TypeScript Server Package
+
+```bash
+cd packages/server
+pnpm dev          # Watch mode with bunchee
+pnpm build        # Multi-step build: ESM/CJS + Next.js + static assets
+pnpm clean        # Clean all build outputs
+```
+
+### Python Server Package
+
+```bash
+cd python/llama-index-server
+uv run generate   # Index data files
+fastapi dev       # Start development server with hot reload
+pytest            # Run test suite
+```
+
+## Template System
+
+The CLI uses a sophisticated template system in `packages/create-llama/templates/`:
+
+### Organization
+
+- **`types/`**: Base project structures (streaming, reflex, llamaindexserver)
+- **`components/`**: Reusable components across frameworks
+  - `engines/` - Chat and agent engines
+  - `loaders/` - File, web, database loaders
+  - `providers/` - AI model configurations
+  - `vectordbs/` - Vector database integrations
+  - `use-cases/` - Workflow implementations
+
+### Development Workflow
+
+- Templates support multiple frameworks (Next.js, Express, FastAPI)
+- Component system allows mix-and-match functionality
+- E2E tests validate generated projects work correctly
+
+## Server Framework Architecture
+
+### TypeScript Server (`@llamaindex/server`)
+
+- **Core**: `LlamaIndexServer` class wrapping Next.js with workflow support
+- **Frontend**: React-based chat UI with shadcn/ui components
+- **API**: `/api/chat` endpoint with streaming responses
+- **Build Process**: Complex multi-step build including static assets for Python integration
+
+### Python Server (`llama-index-server`)
+
+- **Core**: `LlamaIndexServer` class extending FastAPI
+- **Architecture**: Workflow factory pattern for stateless request handling
+- **UI Generation**: AI-powered React component generation from Pydantic schemas
+- **Development**: Hot reloading support with dev mode
+
+## Common Patterns
+
+### Workflow Integration
+
+Both server frameworks use factory patterns:
+
+```typescript
+// TypeScript
+const server = new LlamaIndexServer({
+  workflow: (context) => createWorkflow(context)
+});
+
+// Python
+def create_workflow(chat_request: ChatRequest) -> Workflow:
+    return MyWorkflow(chat_request.messages)
+```
+
+### Event System
+
+Structured events for UI communication:
+
+- **UIEvent**: Custom components with Pydantic/Zod schemas
+- **ArtifactEvent**: Code/documents for Canvas panel
+- **SourceNodesEvent**: Document sources with metadata
+- **AgentRunEvent**: Tool usage and progress tracking
+
+### File Handling
+
+- Both servers auto-mount `data/` and `output/` directories
+- LlamaCloud integration for remote file access
+- Static file serving through framework-specific methods
+
+## Testing Strategy
+
+### E2E Testing
+
+- Playwright tests in `packages/create-llama/e2e/`
+- Tests both Python and TypeScript generated projects
+- Validates CLI generation and application functionality
+
+### Unit Testing
+
+- Python: pytest with comprehensive API and service tests
+- TypeScript: Integrated testing through build process
+
+## Build Process
+
+### Create-llama CLI
+
+1. TypeScript compilation with bash script
+2. ncc bundling for standalone executable
+3. Template validation and caching
+
+### Server Package Build
+
+1. **prebuild**: Clean directories
+2. **build**: bunchee compilation to ESM/CJS
+3. **postbuild**: Next.js preparation and static asset generation
+4. **prepare:py-static**: Python integration assets
+
+### Release Process
+
+```bash
+pnpm release     # Build all + publish npm packages + Python release
+```
+
+## Development Environment Setup
+
+### Prerequisites
+
+- Node.js >=16.14.0
+- Python with uv package manager
+- pnpm for package management
+
+### Common Workflow
+
+1. Clone repository and run `pnpm install`
+2. For CLI development: work in `packages/create-llama/`
+3. For server development: choose TypeScript or Python package
+4. Use `pnpm dev` for concurrent development across packages
+5. Run `pnpm e2e` to validate changes with generated projects
+
+## Special Considerations
+
+### Template Development
+
+- Changes to templates require rebuilding CLI
+- E2E tests validate template functionality across frameworks
+- Template caching system speeds up repeated builds
+
+### Cross-package Dependencies
+
+- Server package builds static assets for Python integration
+- Version synchronization between TypeScript and Python packages
+- Shared UI components and styling across implementations
+
+### Performance
+
+- CLI uses caching for template operations
+- Server frameworks support streaming responses
+- Background processing for file operations and LlamaCloud integration

eslint.config.mjs

@@ -57,6 +57,9 @@ export default tseslint.config(
       "**/out/**",
       "**/node_modules/**",
       "**/build/**",
+      "packages/server/server/**",
+      "packages/server/project/**",
+      "packages/server/bin/**",
     ],
   },
 );

packages/create-llama/CHANGELOG.md

@@ -1,5 +1,24 @@
 # create-llama
 
+## 0.5.20
+
+### Patch Changes
+
+- 3ff0a18: fix: default header padding
+
+## 0.5.19
+
+### Patch Changes
+
+- 5fe9e17: support eject to fully customize next folder
+- b8a1ff6: Support citation for agentic template (Python)
+
+## 0.5.18
+
+### Patch Changes
+
+- 8d59ef0: Add layout_dir config to the generated python code
+
 ## 0.5.17
 
 ### Patch Changes

packages/create-llama/CLAUDE.md

@@ -0,0 +1,108 @@
+# create-llama Package
+
+## Overview
+
+The `create-llama` package is a CLI tool for creating LlamaIndex-powered applications with one command. It's designed as a project generator that scaffolds various types of RAG (Retrieval-Augmented Generation) applications using different frameworks, databases, and AI model providers.
+
+## Package Structure
+
+### Core Files
+
+- **`index.ts`**: Main CLI entry point using Commander.js for argument parsing
+- **`create-app.ts`**: Core application creation logic and orchestration
+- **`package.json`**: Package configuration with binary entry point at `./dist/index.js`
+
+### Key Directories
+
+- **`helpers/`**: Utility functions for package management, file operations, and configuration
+- **`questions/`**: Interactive prompts for user configuration
+- **`templates/`**: Project templates for different frameworks and use cases
+- **`e2e/`**: End-to-end tests using Playwright
+
+## Core Functionality
+
+### CLI Interface
+
+The tool accepts numerous command-line options including:
+
+- Framework selection (`--framework`: nextjs, express, fastapi)
+- Template type (`--template`: streaming, multiagent, reflex, llamaindexserver)
+- Model providers (OpenAI, Anthropic, Groq, Ollama, etc.)
+- Vector databases (none, mongo, pg, pinecone, milvus, etc.)
+- Data sources (files, web URLs, databases)
+- Tools and observability options
+
+### Application Generation Flow
+
+1. **Project validation**: Checks project name validity and directory permissions
+2. **Interactive questioning**: Prompts user for configuration if not provided via CLI
+3. **Template installation**: Copies and configures appropriate templates
+4. **Environment setup**: Creates `.env` files with API keys and configuration
+5. **Dependencies**: Installs packages using detected/specified package manager
+6. **Post-install actions**: Can run the app, open VSCode, or install dependencies
+
+### Template System
+
+Templates are organized by:
+
+- **Framework**: NextJS (frontend), Express (Node backend), FastAPI (Python backend)
+- **Type**: Streaming chat, multiagent workflows, Reflex UI, LlamaIndex server
+- **Components**: Engines, loaders, providers, UI components, observability
+
+### Helper Functions
+
+Key helper modules include:
+
+- **Installation**: Package manager detection and dependency installation
+- **Data sources**: File copying, web scraping, database connection setup
+- **Providers**: Model provider configuration (OpenAI, Anthropic, etc.)
+- **Tools**: Integration with external tools (Wikipedia, weather, code generation)
+- **Environment**: `.env` file generation with API keys and settings
+
+## Development Commands
+
+### Build & Development
+
+- `npm run build`: Build the CLI using bash script
+- `npm run dev`: Watch mode development build
+- `npm run clean`: Clean build artifacts and temporary files
+
+### Testing
+
+- `npm run e2e`: Run all end-to-end tests
+- `npm run e2e:python`: Test Python-specific templates
+- `npm run e2e:typescript`: Test TypeScript-specific templates
+
+### Package Management
+
+- `npm run pack-install`: Create and install local package for testing
+
+## Architecture Notes
+
+### Model Configuration
+
+The tool supports multiple AI providers with a unified `ModelConfig` interface that includes:
+
+- Provider selection and API key management
+- Model and embedding model specification
+- Dimension configuration for embeddings
+
+### Data Source Handling
+
+Flexible data source configuration supporting:
+
+- Local files and directories
+- Web URLs with configurable crawling depth
+- Database connections with custom queries
+- Automatic file downloading and copying
+
+### Template Flexibility
+
+Templates use a component-based system allowing mix-and-match of:
+
+- Different frameworks (NextJS, Express, FastAPI)
+- Various vector databases
+- Multiple observability tools
+- Configurable tools and integrations
+
+This package serves as the foundation for rapidly prototyping and deploying LlamaIndex applications across different technology stacks and use cases.

packages/create-llama/e2e/shared/llamaindexserver_template.spec.ts

@@ -1,5 +1,5 @@
 import { expect, test } from "@playwright/test";
-import { ChildProcess } from "child_process";
+import { ChildProcess, execSync } from "child_process";
 import fs from "fs";
 import path from "path";
 import type {
@@ -28,6 +28,7 @@ const templateUseCases = [
   "deep_research",
   "code_generator",
 ];
+const ejectDir = "next";
 
 for (const useCase of templateUseCases) {
   test.describe(`Test use case ${useCase} ${templateFramework} ${dataSource} ${templateUI} ${appType} ${templatePostInstallAction}`, async () => {
@@ -110,6 +111,28 @@ for (const useCase of templateUseCases) {
       expect(response.ok()).toBeTruthy();
     });
 
+    test("Should successfully eject, install dependencies and build without errors", async () => {
+      test.skip(
+        templateFramework !== "nextjs" ||
+          useCase !== "code_generator" ||
+          dataSource === "--llamacloud",
+        "Eject test only applies to Next.js framework, code generator use case, and non-llamacloud",
+      );
+
+      // Run eject command
+      execSync("npm run eject", { cwd: path.join(cwd, name) });
+
+      // Verify next directory exists
+      const nextDirExists = fs.existsSync(path.join(cwd, name, ejectDir));
+      expect(nextDirExists).toBeTruthy();
+
+      // Install dependencies in next directory
+      execSync("npm install", { cwd: path.join(cwd, name, ejectDir) });
+
+      // Run build
+      execSync("npm run build", { cwd: path.join(cwd, name, ejectDir) });
+    });
+
     // clean processes
     test.afterAll(async () => {
       appProcess?.kill();

packages/create-llama/helpers/python.ts

@@ -5,6 +5,7 @@ import { parse, stringify } from "smol-toml";
 import terminalLink from "terminal-link";
 import { isUvAvailable, tryUvSync } from "./uv";
 
+import { isCI } from "ci-info";
 import { assetRelocator, copy } from "./copy";
 import { templatesDir } from "./dir";
 import { Tool } from "./tools";
@@ -278,6 +279,19 @@ const getAdditionalDependencies = (
     }
   }
 
+  // If app template is llama-index-server and CI and SERVER_PACKAGE_PATH is set,
+  // add @llamaindex/server to dependencies
+  if (
+    templateType === "llamaindexserver" &&
+    isCI &&
+    process.env.SERVER_PACKAGE_PATH
+  ) {
+    dependencies.push({
+      name: "llama-index-server",
+      version: `@file://${process.env.SERVER_PACKAGE_PATH}`,
+    });
+  }
+
   return dependencies;
 };
 

packages/create-llama/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-llama",
-  "version": "0.5.17",
+  "version": "0.5.20",
   "description": "Create LlamaIndex-powered apps with one command",
   "keywords": [
     "rag",

packages/create-llama/templates/components/ui/layout/header.tsx

@@ -4,7 +4,7 @@ import { Sparkles, Star } from "lucide-react";
 
 export default function Header() {
   return (
-    <div className="flex items-center justify-between px-4 pt-2">
+    <div className="flex items-center justify-between p-2 px-4">
       <div className="flex items-center gap-2">
         <Sparkles className="size-4" />
         <h1 className="font-semibold">LlamaIndex App</h1>

packages/create-llama/templates/components/use-cases/python/agentic_rag/workflow.py

@@ -3,9 +3,12 @@ from typing import Optional
 from app.index import get_index
 from llama_index.core.agent.workflow import AgentWorkflow
 from llama_index.core.settings import Settings
-from llama_index.llms.openai import OpenAI
 from llama_index.server.api.models import ChatRequest
 from llama_index.server.tools.index import get_query_engine_tool
+from llama_index.server.tools.index.citation import (
+    CITATION_SYSTEM_PROMPT,
+    enable_citation,
+)
 
 
 def create_workflow(chat_request: Optional[ChatRequest] = None) -> AgentWorkflow:
@@ -14,9 +17,16 @@ def create_workflow(chat_request: Optional[ChatRequest] = None) -> AgentWorkflow
         raise RuntimeError(
             "Index not found! Please run `uv run generate` to index the data first."
         )
-    query_tool = get_query_engine_tool(index=index)
+    # Create a query tool with citations enabled
+    query_tool = enable_citation(get_query_engine_tool(index=index))
+
+    # Define the system prompt for the agent
+    # Append the citation system prompt to the system prompt
+    system_prompt = """You are a helpful assistant"""
+    system_prompt += CITATION_SYSTEM_PROMPT
+
     return AgentWorkflow.from_tools_or_functions(
         tools_or_functions=[query_tool],
-        llm=Settings.llm or OpenAI(model="gpt-4o-mini"),
-        system_prompt="You are a helpful assistant.",
+        llm=Settings.llm,
+        system_prompt=system_prompt,
     )

packages/create-llama/templates/components/use-cases/typescript/agentic_rag/README-template.md

@@ -41,6 +41,14 @@ curl --location 'localhost:3000/api/chat' \
 --data '{ "messages": [{ "role": "user", "content": "What standards for a letter exist?" }] }'
 ```
 
+## Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+
+```bash
+npm run eject
+```
+
 ## Learn More
 
 To learn more about LlamaIndex, take a look at the following resources:

packages/create-llama/templates/components/use-cases/typescript/code_generator/README-template.md

@@ -42,6 +42,14 @@ curl --location 'localhost:3000/api/chat' \
 --data '{ "messages": [{ "role": "user", "content": "Compare the financial performance of Apple and Tesla" }] }'
 ```
 
+## Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+
+```bash
+npm run eject
+```
+
 ## Learn More
 
 To learn more about LlamaIndex, take a look at the following resources:

packages/create-llama/templates/components/use-cases/typescript/deep_research/README-template.md

@@ -53,6 +53,14 @@ curl --location 'localhost:3000/api/chat' \
 --data '{ "messages": [{ "role": "user", "content": "Compare the financial performance of Apple and Tesla" }] }'
 ```
 
+## Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+
+```bash
+npm run eject
+```
+
 ## Learn More
 
 To learn more about LlamaIndex, take a look at the following resources:

packages/create-llama/templates/components/use-cases/typescript/document_generator/README-template.md

@@ -42,6 +42,14 @@ curl --location 'localhost:3000/api/chat' \
 --data '{ "messages": [{ "role": "user", "content": "Compare the financial performance of Apple and Tesla" }] }'
 ```
 
+## Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+
+```bash
+npm run eject
+```
+
 ## Learn More
 
 To learn more about LlamaIndex, take a look at the following resources:

packages/create-llama/templates/components/use-cases/typescript/financial_report/README-template.md

@@ -41,6 +41,14 @@ curl --location 'localhost:3000/api/chat' \
 --data '{ "messages": [{ "role": "user", "content": "Generate a financial report that compares the financial performance of Apple and Tesla" }] }'
 ```
 
+## Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+
+```bash
+npm run eject
+```
+
 ## Learn More
 
 To learn more about LlamaIndex, take a look at the following resources:

packages/create-llama/templates/components/vectordbs/llamaindexserver/llamacloud/typescript/generate.ts

@@ -1,8 +1,9 @@
+import { OpenAI } from "@llamaindex/openai";
 import { generateEventComponent } from "@llamaindex/server";
 import * as dotenv from "dotenv";
 import "dotenv/config";
 import * as fs from "fs/promises";
-import { LLamaCloudFileService, OpenAI } from "llamaindex";
+import { LLamaCloudFileService } from "llamaindex";
 import * as path from "path";
 import { getIndex } from "./app/data";
 import { initSettings } from "./app/settings";

packages/create-llama/templates/types/llamaindexserver/fastapi/app/settings.py

@@ -8,5 +8,5 @@ from llama_index.llms.openai import OpenAI
 def init_settings():
     if os.getenv("OPENAI_API_KEY") is None:
         raise RuntimeError("OPENAI_API_KEY is missing in environment variables")
-    Settings.llm = OpenAI(model="gpt-4o-mini")
-    Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
+    Settings.llm = OpenAI(model="gpt-4.1")
+    Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-large")

packages/create-llama/templates/types/llamaindexserver/fastapi/main.py

@@ -17,6 +17,7 @@ def create_app():
         ui_config=UIConfig(
             component_dir=COMPONENT_DIR,
             dev_mode=True,  # Please disable this in production
+            layout_dir="layout",
         ),
         logger=logger,
         env="dev",

packages/create-llama/templates/types/llamaindexserver/fastapi/pyproject.toml

@@ -12,7 +12,7 @@ dependencies = [
     "pydantic<2.10",
     "aiostream>=0.5.2,<0.6.0",
     "llama-index-core>=0.12.28,<0.13.0",
-    "llama-index-server>=0.1.16,<0.2.0",
+    "llama-index-server>=0.1.17,<0.2.0",
 ]
 
 [project.optional-dependencies]
@@ -46,6 +46,9 @@ disable_error_code = [ "return-value", "assignment" ]
 module = "app.*"
 ignore_missing_imports = false
 
+[tool.hatch.metadata]
+allow-direct-references = true
+
 [build-system]
 requires = [ "hatchling>=1.24" ]
-build-backend = "hatchling.build"
+build-backend = "hatchling.build"

packages/create-llama/templates/types/llamaindexserver/nextjs/package.json

@@ -6,7 +6,8 @@
     "generate:datasource": "tsx src/generate.ts datasource",
     "generate:ui": "tsx src/generate.ts ui",
     "dev": "nodemon",
-    "start": "tsx src/index.ts"
+    "start": "tsx src/index.ts",
+    "eject": "llamaindex-server eject"
   },
   "dependencies": {
     "@llamaindex/openai": "~0.4.0",

packages/server/.gitignore

@@ -1,5 +1,8 @@
 # server contains Nextjs frontend code (not compiled)
 server/
 
+# the ejected nextjs project
+project/
+
 # temp is the copy of next folder but without API folder, used to build frontend static files
 temp/

packages/server/CHANGELOG.md

@@ -1,5 +1,26 @@
 # @llamaindex/server
 
+## 0.2.6
+
+### Patch Changes
+
+- 3ff0a18: fix: default header padding
+- df10474: fix: missing cursor pointer for button
+- 087c961: Support zod and chat-ui hooks for custom components
+
+## 0.2.5
+
+### Patch Changes
+
+- 058b376: Fix generate script for ejected project
+
+## 0.2.4
+
+### Patch Changes
+
+- 5fe9e17: support eject to fully customize next folder
+- b8a1ff6: Bump version: chat-ui@0.4.6
+
 ## 0.2.3
 
 ### Patch Changes

packages/server/CLAUDE.md

@@ -0,0 +1,160 @@
+# @llamaindex/server Package
+
+This package provides a Next.js-based server framework for running LlamaIndex workflows with both API endpoints and a chat UI interface.
+
+## Overview
+
+The `@llamaindex/server` package (`src/`) allows you to quickly launch LlamaIndex Workflows and Agent Workflows as an API server with an optional sophisticated chat UI. It combines a backend API server with a frontend React interface built on Next.js.
+
+## Key Components
+
+### Core Server (src/server.ts)
+
+- **LlamaIndexServer class**: Main server implementation that wraps Next.js
+- Handles workflow factory initialization and UI configuration
+- Manages custom components and layout directories
+- Creates HTTP server with custom routing for chat API
+- Automatically configures client-side config in `public/config.js`
+
+### Chat Handler (src/handlers/chat.ts)
+
+- **handleChat function**: Processes POST requests to `/api/chat`
+- Converts AI SDK messages to LlamaIndex format
+- Manages workflow execution with abort signals
+- Streams responses back to client with optional question suggestions
+- Handles errors and validation
+
+### Workflow Management (src/utils/workflow.ts)
+
+- **runWorkflow function**: Executes workflows with proper event handling
+- Transforms workflow events (tool calls, source nodes) into UI-friendly formats
+- Downloads LlamaCloud files automatically in background
+- Processes agent events and source annotations
+
+### Event System (src/events.ts)
+
+- **Source Events**: For displaying document/file sources with metadata
+- **Agent Events**: For showing agent tool usage and progress
+- **Artifact Events**: For structured data like code/documents sent to Canvas UI
+- Helper functions for converting LlamaIndex data to UI events
+
+### UI Generation (src/utils/gen-ui.ts)
+
+- **generateEventComponent function**: Uses LLM to auto-generate React components
+- Creates workflow for UI planning, aggregation, and code generation
+- Validates generated components against supported dependencies
+- Supports shadcn/ui, lucide-react, tailwind CSS, and LlamaIndex chat-ui
+
+### Types (src/types.ts)
+
+- **WorkflowFactory**: Function signature for creating workflow instances
+- **UIConfig**: Configuration options for chat interface
+- **LlamaIndexServerOptions**: Main server configuration interface
+
+## Next.js Frontend
+
+The `next/` directory contains the React frontend:
+
+### API Routes
+
+- `/api/chat/route.ts`: Main chat endpoint (delegates to handleChat)
+- `/api/components/route.ts`: Serves custom UI components
+- `/api/layout/route.ts`: Serves custom layout components
+- `/api/files/[...slug]/route.ts`: File serving for data/output folders
+
+### UI Components
+
+- Chat interface with message history, streaming responses, and canvas panel
+- Extensible component system for custom workflow events
+- Custom layout support for headers/footers
+- Built with shadcn/ui components and Tailwind CSS
+
+## Build Process
+
+### Development
+
+```bash
+pnpm dev  # Watch mode with bunchee
+```
+
+### Production Build
+
+```bash
+pnpm build  # Multi-step build process
+```
+
+The build process:
+
+1. **prebuild**: Cleans dist, server, and temp directories
+2. **build**: Compiles source with bunchee to ESM/CJS
+3. **postbuild**: Prepares TypeScript server and Python static assets
+4. **prepare:ts-server**: Copies Next.js app, builds CSS, compiles API routes
+5. **prepare:py-static**: Creates static build for Python integration
+
+## Key Features
+
+### Workflow Integration
+
+- Factory pattern for creating workflow instances per request
+- Supports Agent Workflows with startAgentEvent/stopAgentEvent contract
+- Automatic event transformation and streaming
+- Built-in tool call and source node handling
+
+### UI Extensibility
+
+- AI-generated components based on Zod schemas
+- Custom layout sections (header/footer)
+- Canvas panel for artifacts (documents, code)
+- Event aggregation and real-time updates
+
+### File Handling
+
+- Automatic mounting of `data/` and `output/` folders
+- LlamaCloud file downloads in background
+- Static asset serving through Next.js
+
+### Development Features
+
+- Hot reload support for workflow code (beta)
+- Dev mode panel for live code editing
+- TypeScript support throughout
+- Comprehensive error handling
+
+## Configuration
+
+Server configuration through `LlamaIndexServerOptions`:
+
+- `workflow`: Factory function for creating workflow instances
+- `uiConfig.starterQuestions`: Predefined questions for chat interface
+- `uiConfig.componentsDir`: Directory for custom event components
+- `uiConfig.layoutDir`: Directory for custom layout components
+- `uiConfig.llamaCloudIndexSelector`: Enable LlamaCloud integration
+- `uiConfig.devMode`: Enable live code editing
+- `suggestNextQuestions`: Auto-suggest follow-up questions
+
+## Dependencies
+
+### Runtime Dependencies
+
+- Next.js 15+ for server framework
+- React 19+ for UI components
+- LlamaIndex workflow engine
+- Radix UI components (shadcn/ui)
+- AI SDK for streaming responses
+
+### Development Dependencies
+
+- Bunchee for bundling
+- TypeScript for type safety
+- Tailwind CSS for styling
+- PostCSS for CSS processing
+
+## Usage Patterns
+
+1. **Basic Setup**: Create workflow factory, configure UI, start server
+2. **Custom Events**: Define Zod schemas, generate UI components with LLM
+3. **File Integration**: Use data/output folders for document processing
+4. **Development**: Use dev mode for iterative workflow development
+5. **Production**: Build static assets for deployment with Python backend
+
+The package serves as a complete solution for deploying LlamaIndex workflows with professional chat interfaces and extensible UI components.

packages/server/README.md

@@ -300,6 +300,23 @@ The server always provides a chat interface at the root path (`/`) with:
 - The server automatically mounts the `data` and `output` folders at `{server_url}{api_prefix}/files/data` (default: `/api/files/data`) and `{server_url}{api_prefix}/files/output` (default: `/api/files/output`) respectively.
 - Your workflows can use both folders to store and access files. By convention, the `data` folder is used for documents that are ingested, and the `output` folder is used for documents generated by the workflow.
 
+### Eject Mode
+
+If you want to fully customize the server UI and routes, you can use `npm eject`. It will create a normal Next.js project with the same functionality as @llamaindex/server.
+By default, the ejected project will be in the `next` directory in the current working directory. You can change the output directory by providing custom path after `eject` command:
+
+```bash
+npm eject <path-to-output-directory>
+```
+
+How eject works:
+
+1. Init nextjs project with eslint, prettier, postcss, tailwindcss, shadcn components, etc.
+2. Copy your workflow definition and setting files in src/app/\* to the ejected project in app/api/chat
+3. Copy your components, data, output, storage folders to the ejected project
+4. Copy your current .env file to the ejected project
+5. Clean up files that are no longer needed and update imports
+
 ## API Reference
 
 - [LlamaIndexServer](https://ts.llamaindex.ai/docs/api/classes/LlamaIndexServer)

packages/server/bin/eject.cjs

@@ -0,0 +1,172 @@
+#!/usr/bin/env node
+
+const fs = require("fs").promises;
+const path = require("path");
+
+// Resolve the project directory in node_modules/@llamaindex/server/project
+// This is the template that used to construct the nextjs project
+const projectDir = path.resolve(__dirname, "../project");
+
+// Resolve the src directory that contains workflow & setting files
+const srcDir = path.join(process.cwd(), "src");
+const srcAppDir = path.join(srcDir, "app");
+const generateFile = path.join(srcDir, "generate.ts");
+const envFile = path.join(process.cwd(), ".env");
+
+// The environment variables that are used as LlamaIndexServer configs
+const SERVER_CONFIG_VARS = [
+  {
+    key: "OPENAI_API_KEY",
+    defaultValue: "<your-openai-api-key>",
+    description: "OpenAI API key",
+  },
+  {
+    key: "SUGGEST_NEXT_QUESTIONS",
+    defaultValue: "true",
+    description: "Whether to suggest next questions (`suggestNextQuestions`)",
+  },
+  {
+    key: "COMPONENTS_DIR",
+    defaultValue: "components",
+    description: "Directory for custom components (`componentsDir`)",
+  },
+  {
+    key: "WORKFLOW_FILE_PATH",
+    defaultValue: "app/api/chat/app/workflow.ts",
+    description: "The path to the workflow file (will be updated in dev mode)",
+  },
+  {
+    key: "NEXT_PUBLIC_USE_COMPONENTS_DIR",
+    defaultValue: "true",
+    description: "Whether to enable components directory feature on frontend",
+  },
+  {
+    key: "NEXT_PUBLIC_DEV_MODE",
+    defaultValue: "true",
+    description: "Whether to enable dev mode (`devMode`)",
+  },
+  {
+    key: "NEXT_PUBLIC_STARTER_QUESTIONS",
+    defaultValue: '["Summarize the document", "What are the key points?"]',
+    description:
+      "Initial questions to display in the chat (`starterQuestions`)",
+  },
+  {
+    key: "NEXT_PUBLIC_SHOW_LLAMACLOUD_SELECTOR",
+    defaultValue: "false",
+    description:
+      "Whether to show LlamaCloud selector for frontend (`llamaCloudIndexSelector`)",
+  },
+];
+
+async function eject() {
+  try {
+    // validate required directories (nextjs project template, src directory, src/app directory)
+    const requiredDirs = [projectDir, srcDir, srcAppDir];
+    for (const dir of requiredDirs) {
+      const exists = await fs
+        .access(dir)
+        .then(() => true)
+        .catch(() => false);
+      if (!exists) {
+        console.error("Error: directory does not exist at", dir);
+        process.exit(1);
+      }
+    }
+
+    // Get destination directory from command line arguments (pnpm eject <path>)
+    const args = process.argv;
+    const outputIndex = args.indexOf("eject");
+    const destDir =
+      outputIndex !== -1 && args[outputIndex + 1]
+        ? path.resolve(args[outputIndex + 1]) // Use provided path after eject
+        : path.join(process.cwd(), "next"); // Default to "next" folder in the current working directory
+
+    // remove destination directory if it exists
+    await fs.rm(destDir, { recursive: true, force: true });
+
+    // create destination directory
+    await fs.mkdir(destDir, { recursive: true });
+
+    // Copy the nextjs project template to the destination directory
+    await fs.cp(projectDir, destDir, { recursive: true });
+
+    // copy src/app/* to destDir/app/api/chat
+    const chatRouteDir = path.join(destDir, "app", "api", "chat");
+    await fs.cp(srcAppDir, path.join(chatRouteDir, "app"), { recursive: true });
+
+    // nextjs project doesn't depend on @llamaindex/server anymore, we need to update the imports in workflow file
+    const workflowFile = path.join(chatRouteDir, "app", "workflow.ts");
+    let workflowContent = await fs.readFile(workflowFile, "utf-8");
+    workflowContent = workflowContent.replace("@llamaindex/server", "../utils");
+    await fs.writeFile(workflowFile, workflowContent);
+
+    // copy generate.ts if it exists
+    const genFilePath = path.join(chatRouteDir, "generate.ts");
+    const genFileExists = await copy(generateFile, genFilePath);
+    if (genFileExists) {
+      // update the import @llamaindex/server in generate.ts
+      let genContent = await fs.readFile(genFilePath, "utf-8");
+      genContent = genContent.replace("@llamaindex/server", "./utils");
+      await fs.writeFile(genFilePath, genContent);
+    }
+
+    // copy folders in root directory if exists
+    const rootFolders = ["components", "data", "output", "storage"];
+    for (const folder of rootFolders) {
+      await copy(path.join(process.cwd(), folder), path.join(destDir, folder));
+    }
+
+    // copy .env if it exists or create a new one
+    const envFileExists = await copy(envFile, path.join(destDir, ".env"));
+    if (!envFileExists) {
+      await fs.writeFile(path.join(destDir, ".env"), "");
+    }
+
+    // update .env file with more server configs
+    let envFileContent = await fs.readFile(path.join(destDir, ".env"), "utf-8");
+    for (const envVar of SERVER_CONFIG_VARS) {
+      const { key, defaultValue, description } = envVar;
+      if (!envFileContent.includes(key)) {
+        // if the key is not exists in the env file, add it
+        envFileContent += `\n# ${description}\n${key}=${defaultValue}\n`;
+      }
+    }
+    await fs.writeFile(path.join(destDir, ".env"), envFileContent);
+
+    // rename gitignore -> .gitignore
+    await fs.rename(
+      path.join(destDir, "gitignore"),
+      path.join(destDir, ".gitignore"),
+    );
+
+    // user can customize layout directory in nextjs project, remove layout api
+    await fs.rm(path.join(destDir, "app", "api", "layout"), {
+      recursive: true,
+      force: true,
+    });
+
+    // remove no-needed files
+    await fs.unlink(path.join(destDir, "public", "config.js"));
+    await fs.unlink(path.join(destDir, "next-build.config.ts"));
+
+    console.log("Successfully ejected @llamaindex/server to", destDir);
+  } catch (error) {
+    console.error("Error during eject:", error.message);
+    process.exit(1);
+  }
+}
+
+// copy src to dest if src exists, return true if src exists
+async function copy(src, dest) {
+  const srcExists = await fs
+    .access(src)
+    .then(() => true)
+    .catch(() => false);
+  if (srcExists) {
+    await fs.cp(src, dest, { recursive: true });
+  }
+  return srcExists;
+}
+
+eject();

packages/server/examples/CLAUDE.md

@@ -0,0 +1,186 @@
+# LlamaIndex Server Examples
+
+This package contains practical examples demonstrating how to use the `@llamaindex/server` package to build chat applications with LlamaIndex workflows.
+
+## Package Overview
+
+The examples package is a collection of standalone TypeScript applications that showcase different features and capabilities of the LlamaIndex Server framework. Each example can be run independently to demonstrate specific functionality.
+
+## Key Features Demonstrated
+
+### 1. Simple Workflow (`simple-workflow/calculator.ts`)
+
+- **Purpose**: Basic agent workflow with tool integration
+- **Features**: Calculator agent with add tool, starter questions
+- **Key Concepts**: Tool definition with Zod schemas, basic server setup
+
+### 2. Agentic RAG (`agentic-rag/index.ts`)
+
+- **Purpose**: Retrieval-Augmented Generation with document querying
+- **Features**: Vector store index, document ingestion, query engine tool, automatic question suggestions
+- **Key Concepts**: RAG implementation, source node inclusion, embedding models
+
+### 3. Custom Layout (`custom-layout/index.ts` + `layout/header.tsx`)
+
+- **Purpose**: Custom UI components and layout customization
+- **Features**: Weather agent with custom header layout, branded interface
+- **Key Concepts**: Layout directory configuration, React component integration
+
+### 4. Development Mode (`devmode/index.ts` + `src/app/workflow.ts`)
+
+- **Purpose**: Live development and hot reloading capabilities
+- **Features**: Dev mode panel, workflow file hot reloading, separate workflow file structure
+- **Key Concepts**: Development workflow, file watching, modular architecture
+
+## Development Scripts
+
+```bash
+# Type checking
+pnpm typecheck
+
+# Run development server (defaults to simple-workflow/calculator.ts)
+pnpm dev
+
+# Run specific examples
+npx nodemon --exec tsx agentic-rag/index.ts
+npx nodemon --exec tsx custom-layout/index.ts
+npx nodemon --exec tsx devmode/index.ts --ignore src/app/workflow_*.ts  # Dev mode with file watching
+```
+
+## Environment Setup
+
+All examples require OpenAI API access:
+
+```bash
+export OPENAI_API_KEY=your_openai_api_key
+```
+
+## Dependencies
+
+### Core Dependencies
+
+- `@llamaindex/server`: Main server framework (workspace dependency)
+- `@llamaindex/workflow`: Workflow engine for agent creation
+- `@llamaindex/openai`: OpenAI LLM and embedding integrations
+- `@llamaindex/tools`: Tool utilities
+- `@llamaindex/readers`: Document readers
+- `llamaindex`: Core LlamaIndex library
+- `zod`: Schema validation for tools
+
+### Development Dependencies
+
+- `tsx`: TypeScript execution for development
+- `nodemon`: File watching and auto-restart
+- `typescript`: TypeScript compiler
+
+## Architecture Patterns
+
+### Workflow Factory Pattern
+
+All examples use the workflow factory pattern:
+
+```typescript
+const workflowFactory = () => agent({ tools: [...] });
+// or
+const workflowFactory = async () => { /* setup logic */ return agent({ tools: [...] }); };
+```
+
+### Server Configuration
+
+Standard server setup pattern:
+
+```typescript
+new LlamaIndexServer({
+  workflow: workflowFactory,
+  uiConfig: {
+    /* UI configuration */
+  },
+  port: 3000,
+}).start();
+```
+
+### Tool Definition Pattern
+
+Consistent tool creation with Zod schemas:
+
+```typescript
+tool({
+  name: "tool_name",
+  description: "Tool description",
+  parameters: z.object({
+    /* parameters */
+  }),
+  execute: (params) => {
+    /* implementation */
+  },
+});
+```
+
+## Example-Specific Features
+
+### Simple Workflow
+
+- Basic arithmetic operations
+- Minimal setup for learning
+- Demonstrates core workflow concepts
+
+### Agentic RAG
+
+- Document indexing with embeddings
+- Vector similarity search
+- Source node tracking for citations
+- Auto-generated follow-up questions
+
+### Custom Layout
+
+- Custom React components in `layout/` directory
+- Branded header with navigation
+- Layout directory configuration (`layoutDir: "layout"`)
+
+### Dev Mode
+
+- Live code editing in browser
+- Hot reloading of workflow files
+- Separate workflow file organization
+- Development panel UI
+
+## TypeScript Configuration
+
+- Target: ES2022 with bundler module resolution
+- Strict type checking enabled
+- Excludes: `node_modules`, `dist`, `custom-layout/layout` (runtime components)
+- Output: `dist/` directory
+
+## Development Workflow
+
+1. **Choose Example**: Select appropriate example for your use case
+2. **Environment Setup**: Configure OpenAI API key
+3. **Run Development Server**: Use `pnpm dev` or specific nodemon commands
+4. **Access UI**: Open browser at `http://localhost:3000`
+5. **Iterate**: Modify code and see changes in real-time
+
+## Common Patterns
+
+### Agent Creation
+
+All examples use the `agent()` function from `@llamaindex/workflow` with tool arrays.
+
+### UI Configuration
+
+- `starterQuestions`: Predefined questions for user guidance
+- `layoutDir`: Custom layout components directory
+- `devMode`: Enable development features
+- `suggestNextQuestions`: Auto-generate follow-up questions
+
+### Error Handling
+
+Examples demonstrate proper async/await patterns and error handling for LLM operations.
+
+## Integration Points
+
+- **LlamaIndex Core**: Document processing, indexing, querying
+- **OpenAI**: LLM and embedding model integration
+- **React/Next.js**: Frontend UI components and server-side rendering
+- **TypeScript**: Type safety throughout the application stack
+
+This examples package serves as a comprehensive reference for building production-ready chat applications with LlamaIndex workflows.

packages/server/examples/custom-layout/layout/header.tsx

@@ -4,7 +4,7 @@ import { Sparkles, Star } from "lucide-react";
 
 export default function Header() {
   return (
-    <div className="flex items-center justify-between px-4 pt-2">
+    <div className="flex items-center justify-between p-2 px-4">
       <div className="flex items-center gap-2">
         <Sparkles className="size-4" />
         <h1 className="font-semibold">LlamaIndex App</h1>

packages/server/next/README.md

@@ -0,0 +1,45 @@
+This is a [LlamaIndex](https://www.llamaindex.ai/) project using [Next.js](https://nextjs.org/) that is ejected from [`llamaindex-server`](https://github.com/run-llama/create-llama/tree/main/packages/server) via `npm eject` command.
+
+## Quick Start
+
+As this is a Next.js project, you can use the following commands to start the development server:
+
+```bash
+npm install
+npm run dev
+```
+
+Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
+
+## Useful Commands
+
+- Generate Datasource (in case you're having a `./data` folder): `npm run generate`
+- Typecheck: `npm run typecheck`
+- Lint: `npm run lint`
+- Format: `npm run format`
+- Build & Start: `npm run build && npm run start`
+
+## Deployment
+
+The project can be deployed to any platform that supports Next.js like Vercel.
+
+## Configuration
+
+Your original [`llamaindex-server`](https://github.com/run-llama/create-llama/tree/main/packages/server#configuration-options) configurations have been migrated to a [`.env`](.env) file.
+
+Changing the `.env` file will change the behavior of the application, e.g. for changing the initial questions to display in the chat, you can do:
+
+```
+NEXT_PUBLIC_STARTER_QUESTIONS=['What is the capital of France?']
+```
+
+Alternatively, you can also change the file referencing `process.env.NEXT_PUBLIC_STARTER_QUESTIONS` directly in the source code.
+
+## Learn More
+
+To learn more about LlamaIndex, take a look at the following resources:
+
+- [LlamaIndex Documentation](https://docs.llamaindex.ai) - learn about LlamaIndex (Python features).
+- [LlamaIndexTS Documentation](https://ts.llamaindex.ai) - learn about LlamaIndex (Typescript features).
+
+You can check out [the LlamaIndexTS GitHub repository](https://github.com/run-llama/LlamaIndexTS) - your feedback and contributions are welcome!

packages/server/next/app/api/chat/route.ts

@@ -4,16 +4,23 @@ import { type MessageType } from "llamaindex";
 import { NextRequest, NextResponse } from "next/server";
 
 // import chat utils
-import { toDataStream } from "./utils/stream";
-import { sendSuggestedQuestionsEvent } from "./utils/suggestion";
-import { runWorkflow } from "./utils/workflow";
+import {
+  runWorkflow,
+  sendSuggestedQuestionsEvent,
+  toDataStream,
+} from "./utils";
 
-// import workflow factory from local file
-import { workflowFactory } from "../../../../app/workflow";
+// import workflow factory and settings from local file
+import { initSettings } from "./app/settings";
+import { workflowFactory } from "./app/workflow";
+
+initSettings();
 
 export async function POST(req: NextRequest) {
   try {
     const reqBody = await req.json();
+    const suggestNextQuestions = process.env.SUGGEST_NEXT_QUESTIONS === "true";
+
     const { messages } = reqBody as { messages: Message[] };
     const chatHistory = messages.map((message) => ({
       role: message.role as MessageType,
@@ -47,14 +54,15 @@ export async function POST(req: NextRequest) {
     );
 
     const dataStream = toDataStream(workflowEventStream, {
-      // TODO: Support enable/disable suggestion
       callbacks: {
         onFinal: async (completion, dataStreamWriter) => {
           chatHistory.push({
             role: "assistant" as MessageType,
             content: completion,
           });
-          await sendSuggestedQuestionsEvent(dataStreamWriter, chatHistory);
+          if (suggestNextQuestions) {
+            await sendSuggestedQuestionsEvent(dataStreamWriter, chatHistory);
+          }
         },
       },
     });

packages/server/next/app/api/components/route.ts

@@ -3,6 +3,7 @@ import { handleComponentRoute } from "../shared/component-handler";
 
 export async function GET(request: NextRequest) {
   const params = request.nextUrl.searchParams;
-  const directory = params.get("componentsDir") || "components";
+  const directory =
+    params.get("componentsDir") || process.env.COMPONENTS_DIR || "components";
   return handleComponentRoute(directory);
 }

packages/server/next/app/api/dev/files/workflow/route.ts

@@ -4,7 +4,8 @@ import { NextRequest, NextResponse } from "next/server";
 import path from "path";
 import { promisify } from "util";
 
-const DEFAULT_WORKFLOW_FILE_PATH = "src/app/workflow.ts"; // TODO: we can make it as a parameter in server later
+const DEFAULT_WORKFLOW_FILE_PATH =
+  process.env.WORKFLOW_FILE_PATH || "src/app/workflow.ts";
 
 export async function GET(request: NextRequest) {
   const filePath = DEFAULT_WORKFLOW_FILE_PATH;

packages/server/next/app/components/ui/calendar.tsx

@@ -60,12 +60,12 @@ function Calendar({
         ...classNames,
       }}
       components={{
-        IconLeft: ({ className, ...props }) => (
-          <ChevronLeft className={cn("size-4", className)} {...props} />
-        ),
-        IconRight: ({ className, ...props }) => (
-          <ChevronRight className={cn("size-4", className)} {...props} />
-        ),
+        Chevron: ({ ...props }) =>
+          props.orientation === "left" ? (
+            <ChevronLeft {...props} className="h-4 w-4" />
+          ) : (
+            <ChevronRight {...props} className="h-4 w-4" />
+          ),
       }}
       {...props}
     />

packages/server/next/app/components/ui/chat/chat-input.tsx

@@ -8,7 +8,11 @@ import { LlamaCloudSelector } from "./custom/llama-cloud-selector";
 export default function CustomChatInput() {
   const { requestData, isLoading, input } = useChatUI();
   const uploadAPI = getConfig("UPLOAD_API") ?? "";
-  const llamaCloudAPI = getConfig("LLAMA_CLOUD_API") ?? "";
+  const llamaCloudAPI =
+    getConfig("LLAMA_CLOUD_API") ??
+    (process.env.NEXT_PUBLIC_SHOW_LLAMACLOUD_SELECTOR === "true"
+      ? "/api/chat/config/llamacloud"
+      : "");
   const {
     imageUrl,
     setImageUrl,

packages/server/next/app/components/ui/chat/chat-messages.tsx

@@ -32,7 +32,10 @@ export default function CustomChatMessages({
             <ChatMessage.Actions />
           </ChatMessage>
         ))}
-        <ChatMessages.Empty />
+        <ChatMessages.Empty
+          heading="Hello there!"
+          subheading="I'm here to help you with your questions."
+        />
         <ChatMessages.Loading />
       </ChatMessages.List>
       <ChatStarter />

packages/server/next/app/components/ui/chat/chat-section.tsx

@@ -17,7 +17,7 @@ import { ChatLayout } from "./layout";
 
 export default function ChatSection() {
   const handler = useChat({
-    api: getConfig("CHAT_API"),
+    api: getConfig("CHAT_API") || "/api/chat",
     onError: (error: unknown) => {
       if (!(error instanceof Error)) throw error;
       let errorMessage: string;

packages/server/next/app/components/ui/chat/chat-starter.tsx

@@ -6,7 +6,9 @@ import { getConfig } from "../lib/utils";
 
 export function ChatStarter({ className }: { className?: string }) {
   const { append, messages, requestData } = useChatUI();
-  const starterQuestions = getConfig("STARTER_QUESTIONS") ?? [];
+  const starterQuestions =
+    getConfig("STARTER_QUESTIONS") ??
+    JSON.parse(process.env.NEXT_PUBLIC_STARTER_QUESTIONS || "[]");
 
   if (starterQuestions.length === 0 || messages.length > 0) return null;
   return (

packages/server/next/app/components/ui/chat/custom/events/dynamic-events.tsx

@@ -1,7 +1,7 @@
 "use client";
 
 import {
-  getChatUIAnnotation,
+  getAnnotationData,
   JSONValue,
   MessageAnnotation,
   MessageAnnotationType,
@@ -25,9 +25,8 @@ export const DynamicEvents = ({
   componentDefs: ComponentDef[];
   appendError: (error: string) => void;
 }) => {
-  const {
-    message: { annotations },
-  } = useChatMessage();
+  const { message } = useChatMessage();
+  const annotations = message.annotations;
 
   const shownWarningsRef = useRef<Set<string>>(new Set()); // track warnings
   const [hasErrors, setHasErrors] = useState(false);
@@ -43,15 +42,16 @@ export const DynamicEvents = ({
 
     const availableComponents = new Set(componentDefs.map((comp) => comp.type));
 
-    annotations.forEach((annotation: MessageAnnotation) => {
+    annotations.forEach((item: JSONValue) => {
+      const annotation = item as MessageAnnotation;
       const type = annotation.type;
-      if (!type) return; // skip if annotation doesn't have a type
+      if (!type) return; // Skip if annotation doesn't have a type
 
-      const events = getChatUIAnnotation(annotations, type);
+      const events = getAnnotationData<JSONValue>(message, type);
 
       // Skip if it's a built-in component or if we've already shown the warning
       if (
-        BUILT_IN_CHATUI_COMPONENTS.includes(type) ||
+        BUILT_IN_CHATUI_COMPONENTS.includes(type as MessageAnnotationType) ||
         shownWarningsRef.current.has(type)
       ) {
         return;
@@ -69,7 +69,7 @@ export const DynamicEvents = ({
 
   const components: EventComponent[] = componentDefs
     .map((comp) => {
-      const events = getChatUIAnnotation(annotations, comp.type) as JSONValue[]; // get all event data by type
+      const events = getAnnotationData<JSONValue>(message, comp.type);
       if (!events?.length) return null;
       return { ...comp, events };
     })

packages/server/next/app/components/ui/chat/custom/events/import.ts

@@ -67,6 +67,9 @@ export const SOURCE_MAP: Record<string, () => Promise<any>> = {
     import("../../../toggle-group"),
   [`${SHADCN_IMPORT_PREFIX}/tooltip`]: () => import("../../../tooltip"),
 
+  ///// CHAT_UI GENERAL  /////
+  [`@llamaindex/chat-ui`]: () => import("@llamaindex/chat-ui"),
+
   ///// WIDGETS FROM CHAT_UI /////
   [`@llamaindex/chat-ui/widgets`]: () => import("@llamaindex/chat-ui/widgets"),
 
@@ -76,6 +79,9 @@ export const SOURCE_MAP: Record<string, () => Promise<any>> = {
   ///// UTILS /////
   [`@/components/lib/utils`]: () => import("../../../lib/utils"),
   [`@/lib/utils`]: () => import("../../../lib/utils"), // for v0 compatibility
+
+  ///// ZOD /////
+  [`zod`]: () => import("zod"),
 };
 
 // parse imports from code to get Function constructor arguments and component name
@@ -122,7 +128,7 @@ export async function parseImports(code: string) {
   const importPromises = imports.map(async ({ name, source }) => {
     if (!(source in SOURCE_MAP)) {
       throw new Error(
-        `Fail to import ${name} from ${source}. Reason: Module not found. \nCurrently we only support importing UI components from Shadcn components, widgets from "llamaindex/chat-ui/widgets" and icons from "lucide-react"`,
+        `Fail to import ${name} from ${source}. Reason: Module not found. \nCurrently we only support importing UI components from Shadcn components, widgets and hooks from "llamaindex/chat-ui", icons from "lucide-react" and zod for data validation.`,
       );
     }
     try {

packages/server/next/app/components/ui/chat/custom/events/loader.ts

@@ -17,7 +17,11 @@ export async function fetchComponentDefinitions(): Promise<{
   components: ComponentDef[];
   errors: string[];
 }> {
-  const endpoint = getConfig("COMPONENTS_API");
+  const endpoint =
+    getConfig("COMPONENTS_API") ??
+    (process.env.NEXT_PUBLIC_USE_COMPONENTS_DIR === "true"
+      ? "/api/components"
+      : undefined);
   if (!endpoint) {
     console.warn("/api/components endpoint is not defined in config");
     return { components: [], errors: [] };

packages/server/next/app/components/ui/chat/custom/llama-cloud-selector.tsx

@@ -65,8 +65,14 @@ export function LlamaCloudSelector({
   );
 
   useEffect(() => {
-    if (!config && getConfig("LLAMA_CLOUD_API")) {
-      fetch(getConfig("LLAMA_CLOUD_API"))
+    const llamaCloudAPI =
+      getConfig("LLAMA_CLOUD_API") ??
+      (process.env.NEXT_PUBLIC_SHOW_LLAMACLOUD_SELECTOR === "true"
+        ? "/api/chat/config/llamacloud"
+        : "");
+
+    if (!config && llamaCloudAPI) {
+      fetch(llamaCloudAPI)
         .then((response) => {
           if (!response.ok) {
             return response.json().then((errorData) => {

packages/server/next/app/components/ui/chat/custom/markdown.tsx

@@ -1,7 +1,9 @@
 "use client";
 
-import { SourceData } from "@llamaindex/chat-ui";
-import { Markdown as MarkdownUI } from "@llamaindex/chat-ui/widgets";
+import {
+  Markdown as MarkdownUI,
+  SourceData,
+} from "@llamaindex/chat-ui/widgets";
 import { getConfig } from "../../lib/utils";
 const preprocessMedia = (content: string) => {
   // Remove `sandbox:` from the beginning of the URL before rendering markdown

packages/server/next/app/components/ui/chat/dev-mode-panel.tsx

@@ -19,7 +19,8 @@ type WorkflowFile = {
 };
 
 export function DevModePanel() {
-  const devModeEnabled = getConfig("DEV_MODE");
+  const devModeEnabled =
+    getConfig("DEV_MODE") ?? process.env.NEXT_PUBLIC_DEV_MODE === "true";
   if (!devModeEnabled) return null;
   return <DevModePanelComp />;
 }

packages/server/next/app/components/ui/chat/layout/header.tsx

@@ -4,7 +4,7 @@ import { Sparkles, Star } from "lucide-react";
 
 export function DefaultHeader() {
   return (
-    <div className="flex items-center justify-between px-4 pt-2">
+    <div className="flex items-center justify-between p-2 px-4">
       <div className="flex items-center gap-2">
         <Sparkles className="size-4" />
         <h1 className="font-semibold">LlamaIndex App</h1>

packages/server/next/app/components/ui/chat/layout/index.tsx

@@ -121,7 +121,9 @@ async function parseLayoutComponents(layoutFiles: LayoutFile[]) {
 
 async function fetchLayoutFiles(): Promise<LayoutFile[]> {
   try {
-    const response = await fetch(getConfig("LAYOUT_API"));
+    const layoutApi = getConfig("LAYOUT_API");
+    if (!layoutApi) return [];
+    const response = await fetch(layoutApi);
     const layoutFiles: LayoutFile[] = await response.json();
     return layoutFiles;
   } catch (error) {

packages/server/next/app/components/ui/chat/tools/chat-tools.tsx

@@ -2,8 +2,7 @@
 
 import {
   Message,
-  MessageAnnotation,
-  getChatUIAnnotation,
+  getAnnotationData,
   useChatMessage,
   useChatUI,
 } from "@llamaindex/chat-ui";
@@ -21,13 +20,10 @@ export function ToolAnnotations() {
     [messages, message],
   );
   // Get the tool data from the message annotations
-  const annotations = message.annotations as MessageAnnotation[] | undefined;
-  const toolData = annotations
-    ? (getChatUIAnnotation(annotations, "tools") as unknown as ToolData[])
-    : null;
-  return toolData?.[0] ? (
-    <ChatTools data={toolData[0]} artifactVersion={artifactVersion} />
-  ) : null;
+  const toolData = getAnnotationData<ToolData>(message, "tools");
+  if (toolData.length === 0) return null;
+
+  return <ChatTools data={toolData[0]} artifactVersion={artifactVersion} />;
 }
 
 // TODO: Used to render outputs of tools. If needed, add more renderers here.
@@ -83,9 +79,7 @@ function getArtifactVersion(
   if (!messageId) return undefined;
   let versionIndex = 1;
   for (const m of messages) {
-    const toolData = m.annotations
-      ? (getChatUIAnnotation(m.annotations, "tools") as unknown as ToolData[])
-      : null;
+    const toolData = getAnnotationData<ToolData>(m, "tools");
 
     if (toolData?.some((t) => t.toolCall.name === "artifact")) {
       if ("id" in m && m.id === messageId) {

packages/server/next/app/globals.css

@@ -91,6 +91,13 @@
   ::file-selector-button {
     border-color: var(--color-gray-200, currentColor);
   }
+
+  /* Tailwind v4 removed cursor pointer of button and use default cursor */
+  /* https://github.com/shadcn-ui/ui/issues/6843#issuecomment-2696947980 */
+  button:not([disabled]),
+  [role="button"]:not([disabled]) {
+    cursor: pointer;
+  }
 }
 
 @layer base {

packages/server/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/server",
   "description": "LlamaIndex Server",
-  "version": "0.2.3",
+  "version": "0.2.6",
   "type": "module",
   "main": "./dist/index.cjs",
   "module": "./dist/index.js",
@@ -19,8 +19,13 @@
   },
   "files": [
     "dist",
-    "server"
+    "server",
+    "project",
+    "bin"
   ],
+  "bin": {
+    "llamaindex-server": "./bin/eject.cjs"
+  },
   "repository": {
     "type": "git",
     "url": "git+https://github.com/run-llama/LlamaIndexTS.git",
@@ -28,10 +33,11 @@
   },
   "scripts": {
     "dev": "bunchee --watch",
-    "clean": "rm -rf ./dist ./server next/.next next/out ./temp",
+    "clean": "rm -rf ./dist ./server ./project next/.next next/out ./temp",
     "prebuild": "pnpm clean",
     "build": "bunchee",
-    "postbuild": "pnpm prepare:ts-server && pnpm prepare:py-static",
+    "postbuild": "pnpm prepare:nextjs && pnpm prepare:ts-server && pnpm prepare:py-static",
+    "prepare:nextjs": "cp -r ./next ./project && cp -r ./src/utils ./project/app/api/chat && cp -r ./project-config/* ./project/",
     "prepare:ts-server": "pnpm copy:next-src && pnpm build:css && pnpm build:api",
     "prepare:py-static": "pnpm prepare:static && pnpm build:static && pnpm copy:static",
     "copy:next-src": "cp -r ./next ./server",
@@ -59,7 +65,7 @@
     "@babel/traverse": "^7.27.0",
     "@babel/types": "^7.27.0",
     "@hookform/resolvers": "^5.0.1",
-    "@llamaindex/chat-ui": "0.4.5",
+    "@llamaindex/chat-ui": "0.4.9",
     "@radix-ui/react-accordion": "^1.2.3",
     "@radix-ui/react-alert-dialog": "^1.1.7",
     "@radix-ui/react-aspect-ratio": "^1.1.3",
@@ -97,7 +103,7 @@
     "next": "^15.3.0",
     "next-themes": "^0.4.3",
     "react": "^19.1.0",
-    "react-day-picker": "8.10.1",
+    "react-day-picker": "9.7.0",
     "react-dom": "^19.1.0",
     "react-hook-form": "^7.55.0",
     "react-resizable-panels": "^2.1.7",

packages/server/project-config/eslint.config.mjs

@@ -0,0 +1,34 @@
+import { FlatCompat } from "@eslint/eslintrc";
+import { dirname } from "path";
+import { fileURLToPath } from "url";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+
+const compat = new FlatCompat({
+  baseDirectory: __dirname,
+});
+
+const eslintConfig = [
+  ...compat.extends("next/core-web-vitals", "next/typescript", "prettier"),
+  {
+    rules: {
+      "@typescript-eslint/no-explicit-any": "off",
+      "@typescript-eslint/no-unused-vars": "off",
+      "react-hooks/exhaustive-deps": "off",
+      "@next/next/no-img-element": "off",
+      "@next/next/no-assign-module-variable": "off",
+    },
+  },
+  {
+    ignores: [
+      "**/.next/**",
+      "**/node_modules/**",
+      "prettier.config.mjs",
+      "eslint.config.mjs",
+      "postcss.config.js",
+    ],
+  },
+];
+
+export default eslintConfig;

packages/server/project-config/gitignore

@@ -26,6 +26,7 @@ yarn-error.log*
 
 # local env files
 .env*.local
+.env
 
 # vercel
 .vercel
@@ -35,5 +36,6 @@ yarn-error.log*
 next-env.d.ts
 
 output/
+storage/
 
 !lib/

packages/server/project-config/package.json

@@ -0,0 +1,100 @@
+{
+  "name": "nextjs-project",
+  "description": "Next.js project with full feature set of @llamaindex/server",
+  "private": true,
+  "version": "0.0.1",
+  "type": "module",
+  "scripts": {
+    "dev": "next dev",
+    "build": "next build",
+    "start": "next start",
+    "lint": "next lint",
+    "format": "prettier --ignore-unknown --cache --check .",
+    "format:write": "prettier --ignore-unknown --write .",
+    "typecheck": "tsc --noEmit",
+    "generate": "tsx app/api/chat/generate.ts"
+  },
+  "devDependencies": {
+    "@eslint/eslintrc": "^3",
+    "@next/eslint-plugin-next": "^15.3.2",
+    "@tailwindcss/postcss": "^4",
+    "@types/babel__standalone": "^7.1.9",
+    "@types/babel__traverse": "^7.20.7",
+    "@types/node": "^20",
+    "@types/react": "^19",
+    "@types/react-dom": "^19",
+    "eslint": "^9",
+    "eslint-config-next": "^15.1.3",
+    "eslint-config-prettier": "^9.1.0",
+    "eslint-plugin-react-hooks": "^5.2.0",
+    "prettier": "^3.2.5",
+    "prettier-plugin-organize-imports": "^4.1.0",
+    "prettier-plugin-tailwindcss": "^0.6.11",
+    "tailwindcss": "^4",
+    "tsx": "^4.19.3",
+    "tw-animate-css": "1.2.5",
+    "typescript": "^5"
+  },
+  "dependencies": {
+    "@babel/parser": "^7.27.0",
+    "@babel/standalone": "^7.27.0",
+    "@babel/traverse": "^7.27.0",
+    "@babel/types": "^7.27.0",
+    "@hookform/resolvers": "^5.0.1",
+    "@llamaindex/chat-ui": "0.4.9",
+    "@llamaindex/env": "~0.1.30",
+    "@llamaindex/openai": "~0.4.0",
+    "@llamaindex/readers": "~3.1.4",
+    "@llamaindex/tools": "~0.0.11",
+    "@llamaindex/workflow": "~1.1.3",
+    "@radix-ui/react-accordion": "^1.2.3",
+    "@radix-ui/react-alert-dialog": "^1.1.7",
+    "@radix-ui/react-aspect-ratio": "^1.1.3",
+    "@radix-ui/react-avatar": "^1.1.4",
+    "@radix-ui/react-checkbox": "^1.1.5",
+    "@radix-ui/react-collapsible": "^1.1.3",
+    "@radix-ui/react-context-menu": "^2.2.7",
+    "@radix-ui/react-dialog": "^1.1.2",
+    "@radix-ui/react-dropdown-menu": "^2.1.7",
+    "@radix-ui/react-hover-card": "^1.1.7",
+    "@radix-ui/react-label": "^2.1.0",
+    "@radix-ui/react-menubar": "^1.1.7",
+    "@radix-ui/react-navigation-menu": "^1.2.6",
+    "@radix-ui/react-popover": "^1.1.7",
+    "@radix-ui/react-progress": "^1.1.3",
+    "@radix-ui/react-radio-group": "^1.2.4",
+    "@radix-ui/react-scroll-area": "^1.2.4",
+    "@radix-ui/react-select": "^2.1.6",
+    "@radix-ui/react-separator": "^1.1.3",
+    "@radix-ui/react-slider": "^1.2.1",
+    "@radix-ui/react-slot": "^1.1.2",
+    "@radix-ui/react-switch": "^1.1.4",
+    "@radix-ui/react-tabs": "^1.1.3",
+    "@radix-ui/react-toggle": "^1.1.3",
+    "@radix-ui/react-toggle-group": "^1.1.3",
+    "@radix-ui/react-tooltip": "^1.1.4",
+    "ai": "^4.2.0",
+    "class-variance-authority": "^0.7.1",
+    "clsx": "^2.1.1",
+    "cmdk": "^1.1.1",
+    "date-fns": "^4.1.0",
+    "dotenv": "^16.5.0",
+    "embla-carousel-react": "^8.6.0",
+    "input-otp": "^1.4.2",
+    "llamaindex": "~0.11.0",
+    "lucide-react": "^0.460.0",
+    "next": "^15.3.0",
+    "next-themes": "^0.4.3",
+    "react": "^19.1.0",
+    "react-day-picker": "9.7.0",
+    "react-dom": "^19.1.0",
+    "react-hook-form": "^7.55.0",
+    "react-resizable-panels": "^2.1.7",
+    "recharts": "^2.15.2",
+    "sonner": "^2.0.3",
+    "tailwind-merge": "^2.6.0",
+    "vaul": "^1.1.2",
+    "zod": "^3.23.8",
+    "zod-to-json-schema": "^3.23.3"
+  }
+}

packages/server/project-config/prettier.config.mjs

@@ -0,0 +1,3 @@
+export default {
+  plugins: ["prettier-plugin-organize-imports", "prettier-plugin-tailwindcss"],
+};

packages/server/src/index.ts

@@ -1,5 +1,5 @@
-export * from "./events";
-export * from "./prompts";
 export * from "./server";
 export * from "./types";
+export * from "./utils/events";
 export { generateEventComponent } from "./utils/gen-ui";
+export * from "./utils/prompts";

packages/server/src/utils/index.ts

@@ -0,0 +1,8 @@
+export * from "./events";
+export * from "./file";
+export * from "./gen-ui";
+export * from "./prompts";
+export * from "./request";
+export * from "./stream";
+export * from "./suggestion";
+export * from "./workflow";

packages/server/src/utils/suggestion.ts

@@ -1,7 +1,7 @@
 import { getEnv } from "@llamaindex/env";
 import type { DataStreamWriter } from "ai";
 import { type ChatMessage, Settings } from "llamaindex";
-import { NEXT_QUESTION_PROMPT } from "../prompts";
+import { NEXT_QUESTION_PROMPT } from "./prompts";
 
 export const sendSuggestedQuestionsEvent = async (
   streamWriter: DataStreamWriter,

packages/server/src/utils/workflow.ts

@@ -19,7 +19,7 @@ import {
   toAgentRunEvent,
   toSourceEvent,
   type SourceEventNode,
-} from "../events";
+} from "./events";
 import { downloadFile } from "./file";
 
 export async function runWorkflow(

pnpm-lock.yaml

@@ -181,8 +181,8 @@ importers:
         specifier: ^5.0.1
         version: 5.0.1(react-hook-form@7.56.1(react@19.1.0))
       '@llamaindex/chat-ui':
-        specifier: 0.4.5
-        version: 0.4.5(@babel/runtime@7.27.0)(@codemirror/autocomplete@6.18.6)(@codemirror/language@6.11.0)(@codemirror/lint@6.8.5)(@codemirror/search@6.5.10)(@codemirror/state@6.5.2)(@codemirror/theme-one-dark@6.1.2)(@codemirror/view@6.36.7)(@types/react-dom@19.1.2(@types/react@19.1.2))(@types/react@19.1.2)(codemirror@6.0.1)(react-dom@19.1.0(react@19.1.0))(react@19.1.0)
+        specifier: 0.4.9
+        version: 0.4.9(@babel/runtime@7.27.0)(@codemirror/autocomplete@6.18.6)(@codemirror/language@6.11.0)(@codemirror/lint@6.8.5)(@codemirror/search@6.5.10)(@codemirror/state@6.5.2)(@codemirror/theme-one-dark@6.1.2)(@codemirror/view@6.36.7)(@types/react-dom@19.1.2(@types/react@19.1.2))(@types/react@19.1.2)(codemirror@6.0.1)(react-dom@19.1.0(react@19.1.0))(react@19.1.0)
       '@llamaindex/env':
         specifier: ~0.1.30
         version: 0.1.30
@@ -301,8 +301,8 @@ importers:
         specifier: ^19.1.0
         version: 19.1.0
       react-day-picker:
-        specifier: 8.10.1
-        version: 8.10.1(date-fns@4.1.0)(react@19.1.0)
+        specifier: 9.7.0
+        version: 9.7.0(react@19.1.0)
       react-dom:
         specifier: ^19.1.0
         version: 19.1.0(react@19.1.0)
@@ -613,6 +613,9 @@ packages:
     peerDependencies:
       '@bufbuild/protobuf': ^2.2.0
 
+  '@date-fns/tz@1.2.0':
+    resolution: {integrity: sha512-LBrd7MiJZ9McsOgxqWX7AaxrDjcFVjWH/tIKJd7pnR7McaslGYOP1QmmiBXdJH/H/yLCT+rcQ7FaPBUxRGUtrg==}
+
   '@discoveryjs/json-ext@0.6.3':
     resolution: {integrity: sha512-4B4OijXeVNOPZlYA2oEwWOTkzyltLao+xbotHQeqN++Rv27Y6s818+n2Qkp8q+Fxhn0t/5lA5X1Mxktud8eayQ==}
     engines: {node: '>=14.17.0'}
@@ -1186,8 +1189,8 @@ packages:
       zod:
         optional: true
 
-  '@llamaindex/chat-ui@0.4.5':
-    resolution: {integrity: sha512-LWN+w8vJncbMokSaUDsX7/0iGQoVnT3zSemqHc8FM8WwmWGPQ9RPgxygaxrzZxtQueGJtrGLVYfJJmkMFGiN6w==}
+  '@llamaindex/chat-ui@0.4.9':
+    resolution: {integrity: sha512-KEdydC+aJ22VK/TltxIHlMWbWLfh6I0YkyVd1D/CS3FRfLt8l9jfQ/YjY10MiEd8oc1fFfk6ek/FhVWe9Szstg==}
     peerDependencies:
       react: ^18.2.0 || ^19.0.0 || ^19.0.0-rc
 
@@ -3256,6 +3259,9 @@ packages:
     resolution: {integrity: sha512-BS8PfmtDGnrgYdOonGZQdLZslWIeCGFP9tpan0hi1Co2Zr2NKADsvGYA8XxuG/4UWgJ6Cjtv+YJnB6MM69QGlQ==}
     engines: {node: '>= 0.4'}
 
+  date-fns-jalali@4.1.0-0:
+    resolution: {integrity: sha512-hTIP/z+t+qKwBDcmmsnmjWTduxCg+5KfdqWQvb2X/8C9+knYY6epN/pfxdDuyVlSVeFz0sM5eEfwIUQ70U4ckg==}
+
   date-fns@4.1.0:
     resolution: {integrity: sha512-Ukq0owbQXxa/U3EGtsdVBkR1w7KOQ5gIBqdH2hkvknzZPYvBxb/aa6E8L7tmjFtkwZBu3UXBbjIgPo/Ez4xaNg==}
 
@@ -5358,11 +5364,11 @@ packages:
     resolution: {integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==}
     hasBin: true
 
-  react-day-picker@8.10.1:
-    resolution: {integrity: sha512-TMx7fNbhLk15eqcMt+7Z7S2KF7mfTId/XJDjKE8f+IUcFn0l08/kI4FiYTL/0yuOLmEcbR4Fwe3GJf/NiiMnPA==}
+  react-day-picker@9.7.0:
+    resolution: {integrity: sha512-urlK4C9XJZVpQ81tmVgd2O7lZ0VQldZeHzNejbwLWZSkzHH498KnArT0EHNfKBOWwKc935iMLGZdxXPRISzUxQ==}
+    engines: {node: '>=18'}
     peerDependencies:
-      date-fns: ^2.28.0 || ^3.0.0
-      react: ^16.8.0 || ^17.0.0 || ^18.0.0
+      react: '>=16.8.0'
 
   react-dom@19.1.0:
     resolution: {integrity: sha512-Xs1hdnE+DyKgeHJeJznQmYMIBG3TKIHJJT95Q58nHLSrElKlGQqDTR2HQ9fx5CN/Gk6Vh/kupBTDLU11/nDk/g==}
@@ -6812,6 +6818,8 @@ snapshots:
     dependencies:
       '@bufbuild/protobuf': 2.3.0
 
+  '@date-fns/tz@1.2.0': {}
+
   '@discoveryjs/json-ext@0.6.3': {}
 
   '@e2b/code-interpreter@1.5.0':
@@ -7211,7 +7219,7 @@ snapshots:
       p-retry: 6.2.1
       zod: 3.24.3
 
-  '@llamaindex/chat-ui@0.4.5(@babel/runtime@7.27.0)(@codemirror/autocomplete@6.18.6)(@codemirror/language@6.11.0)(@codemirror/lint@6.8.5)(@codemirror/search@6.5.10)(@codemirror/state@6.5.2)(@codemirror/theme-one-dark@6.1.2)(@codemirror/view@6.36.7)(@types/react-dom@19.1.2(@types/react@19.1.2))(@types/react@19.1.2)(codemirror@6.0.1)(react-dom@19.1.0(react@19.1.0))(react@19.1.0)':
+  '@llamaindex/chat-ui@0.4.9(@babel/runtime@7.27.0)(@codemirror/autocomplete@6.18.6)(@codemirror/language@6.11.0)(@codemirror/lint@6.8.5)(@codemirror/search@6.5.10)(@codemirror/state@6.5.2)(@codemirror/theme-one-dark@6.1.2)(@codemirror/view@6.36.7)(@types/react-dom@19.1.2(@types/react@19.1.2))(@types/react@19.1.2)(codemirror@6.0.1)(react-dom@19.1.0(react@19.1.0))(react@19.1.0)':
     dependencies:
       '@codemirror/lang-css': 6.3.1
       '@codemirror/lang-html': 6.4.9
@@ -9443,6 +9451,8 @@ snapshots:
       es-errors: 1.3.0
       is-data-view: 1.0.2
 
+  date-fns-jalali@4.1.0-0: {}
+
   date-fns@4.1.0: {}
 
   debug@3.2.7:
@@ -11895,9 +11905,11 @@ snapshots:
       minimist: 1.2.8
       strip-json-comments: 2.0.1
 
-  react-day-picker@8.10.1(date-fns@4.1.0)(react@19.1.0):
+  react-day-picker@9.7.0(react@19.1.0):
     dependencies:
+      '@date-fns/tz': 1.2.0
       date-fns: 4.1.0
+      date-fns-jalali: 4.1.0-0
       react: 19.1.0
 
   react-dom@19.1.0(react@19.1.0):

python/llama-index-server/CHANGELOG.md

@@ -1,11 +1,35 @@
 # @create-llama/llama-index-server
 
+## 0.1.20
+
+### Patch Changes
+
+- 087c961: Add support for human-in-the-loop
+- 087c961: Refactor models.py into a separate module
+- Updated dependencies [3ff0a18]
+- Updated dependencies [df10474]
+- Updated dependencies [087c961]
+  - @llamaindex/server@0.2.6
+
+## 0.1.19
+
+### Patch Changes
+
+- Updated dependencies [058b376]
+  - @llamaindex/server@0.2.5
+
+## 0.1.18
+
+### Patch Changes
+
+- b8a1ff6: Show agent widget in UI when making a tool call
+- b8a1ff6: Support citation for query engine tool
+- Updated dependencies [5fe9e17]
+- Updated dependencies [b8a1ff6]
+  - @llamaindex/server@0.2.4
+
 ## 0.1.17
 
 ### Patch Changes
 
-- 0bc5a0d: Add suggestNextQuestions config
-- Updated dependencies [eee3230]
-- Updated dependencies [0bc5a0d]
-- Updated dependencies [3acec88]
-  - @llamaindex/server@0.2.3
+- 91c35cf: Add suggestNextQuestions config

python/llama-index-server/CLAUDE.md

@@ -0,0 +1,245 @@
+# LlamaIndex Server (Python)
+
+## Overview
+
+The `llama-index-server` package is a FastAPI-based server framework for deploying LlamaIndex Workflows and Agent Workflows as a high-performance API server with an optional chat UI. It provides a complete environment for running LlamaIndex workflows with both API endpoints and a user interface for interaction.
+
+## Package Structure
+
+### Core Components
+- **`llama_index/server/server.py`**: Main `LlamaIndexServer` class extending FastAPI
+- **`llama_index/server/__init__.py`**: Package exports (`LlamaIndexServer`, `UIConfig`, `UIEvent`)
+- **`pyproject.toml`**: Package configuration with dependencies and build settings
+
+### Key Directories
+- **`api/`**: FastAPI routers, models, and request handling
+- **`services/`**: Business logic for file handling, LlamaCloud integration, and UI generation
+- **`tools/`**: Document generation, interpreter tools, and index querying utilities
+- **`gen_ui/`**: AI-powered UI component generation system
+- **`resources/`**: Static assets and bundled UI files
+- **`examples/`**: Sample workflows demonstrating different features
+
+## Core Functionality
+
+### LlamaIndexServer Class
+Main server implementation that extends FastAPI with workflow-specific features:
+- **Workflow Factory Pattern**: Creates workflow instances per request using factory functions
+- **UI Configuration**: Manages chat interface, custom components, and layout directories
+- **File Serving**: Automatically mounts `data/` and `output/` directories
+- **Development Mode**: Enables CORS, verbose logging, and hot reloading
+
+### Chat API (`api/routers/chat.py`)
+- **Endpoint**: `/api/chat` for chat interactions
+- **Streaming Responses**: Real-time workflow execution with Vercel-compatible streaming
+- **Message Handling**: Converts between API and LlamaIndex message formats
+- **Background Tasks**: File downloads and asynchronous processing
+- **LlamaCloud Integration**: Optional index selector for cloud-based retrieval
+
+### Event System (`api/models.py`)
+Structured event types for workflow communication:
+- **`UIEvent`**: Custom UI component rendering with Pydantic data models
+- **`ArtifactEvent`**: Code and document artifacts for Canvas panel display
+- **`SourceNodesEvent`**: Document sources with metadata and file URLs
+- **`AgentRunEvent`**: Agent tool usage and progress tracking
+
+### UI Generation (`gen_ui/main.py`)
+AI-powered component generation using LLM workflows:
+- **`GenUIWorkflow`**: Multi-step process for creating React components
+- **Planning Phase**: Analyzes event schemas to design UI layouts
+- **Aggregation Logic**: Groups events for optimized rendering
+- **Code Generation**: Creates shadcn/ui components with proper imports
+- **Validation**: Ensures generated code uses only supported dependencies
+
+## Development Environment
+
+### Dependencies
+```toml
+# Core FastAPI server with standard extensions
+fastapi[standard]>=0.115.11,<1.0.0
+
+# LlamaIndex core and workflow engine
+llama-index-core>=0.12.28,<1.0.0
+
+# File handling and cloud integration
+llama-index-readers-file>=0.4.6,<1.0.0
+llama-index-indices-managed-llama-cloud>=0.6.3,<1.0.0
+
+# HTTP requests and caching
+requests>=2.32.3,<3.0.0
+cachetools>=5.5.2,<6.0.0
+pydantic-settings>=2.8.1,<3.0.0
+```
+
+### Development Dependencies
+- **Testing**: pytest, pytest-asyncio, pytest-mock for comprehensive testing
+- **Code Quality**: black, ruff, mypy, pylint for code formatting and linting
+- **Documentation**: jupyter, markdown for examples and documentation
+- **Integrations**: e2b-code-interpreter, llama-cloud for extended functionality
+
+### Build System
+- **Backend**: Hatchling for Python package building
+- **Artifacts**: Includes `llama_index/server/resources` for bundled UI assets
+- **Type Checking**: MyPy with strict settings for type safety
+
+## Configuration Options
+
+### Server Configuration
+```python
+LlamaIndexServer(
+    workflow_factory=create_workflow,  # Required: factory function
+    env="dev",                        # Environment: "dev" enables CORS and UI
+    ui_config={                       # Optional UI configuration
+        "enabled": True,              # Enable chat interface
+        "starter_questions": [...],   # Predefined user prompts
+        "component_dir": "components", # Custom UI components directory
+        "layout_dir": "layout",       # Custom layout sections directory
+        "dev_mode": True,             # Enable live code editing
+        "llamacloud_index_selector": False, # LlamaCloud integration
+    },
+    suggest_next_questions=True,      # Auto-generate follow-up questions
+    verbose=True,                     # Enable detailed logging
+    api_prefix="/api",               # API route prefix
+    server_url="http://localhost:8000", # Deployment URL
+)
+```
+
+### Workflow Factory Contract
+```python
+def create_workflow(chat_request: ChatRequest) -> Workflow:
+    # Access to request information for initialization
+    return MyCustomWorkflow(chat_request.messages)
+
+# Workflow input parameters (StartEvent):
+# - user_msg: str - Current user message
+# - chat_history: List[ChatMessage] - Previous conversation messages
+```
+
+## API Endpoints
+
+### Default Routes
+- **`/api/chat`**: Main chat interaction endpoint with streaming responses
+- **`/api/files/data/*`**: Static file serving from data directory
+- **`/api/files/output/*`**: Generated file serving from output directory
+- **`/api/components`**: Custom UI component serving (if configured)
+- **`/api/layout`**: Custom layout component serving (if configured)
+- **`/api/chat/config/llamacloud`**: LlamaCloud configuration (if enabled)
+
+### Development Routes (Dev Mode)
+- **`/api/dev/*`**: Live code editing and hot reloading endpoints
+
+## UI System
+
+### Chat Interface
+When enabled (`ui_config.enabled=True`), provides:
+- **Real-time Chat**: WebSocket-like streaming with message history
+- **Starter Questions**: Configurable prompts to guide users
+- **Canvas Panel**: Dedicated area for code and document artifacts
+- **Custom Components**: React components for workflow-specific events
+- **Custom Layout**: Configurable header/footer sections
+
+### Component Generation
+Automated UI component creation for workflow events:
+- **Event Analysis**: Parses Pydantic schemas to understand data structure
+- **Design Planning**: LLM generates layout descriptions based on event types
+- **Code Generation**: Creates React components using shadcn/ui and Tailwind CSS
+- **Dependency Validation**: Ensures only supported libraries are used
+
+### Supported UI Dependencies
+- **React**: Core framework with hooks and state management
+- **shadcn/ui**: Complete component library (Button, Card, Table, etc.)
+- **Lucide React**: Icon library for visual elements
+- **Tailwind CSS**: Utility-first styling with `cn` helper
+- **LlamaIndex Chat UI**: Markdown rendering and specialized widgets
+
+## File Handling
+
+### Directory Structure
+```
+project/
+├── data/           # Input documents and ingestion files
+├── output/         # Generated files and workflow outputs
+├── components/     # Custom UI components (optional)
+├── layout/         # Custom layout sections (optional)
+└── .ui/           # Downloaded UI static files
+```
+
+### File Serving
+- **Automatic Mounting**: `data/` and `output/` directories served at `/api/files/`
+- **URL Generation**: Metadata-based file URL creation for source nodes
+- **LlamaCloud Integration**: Background downloading of cloud-hosted files
+- **Static Assets**: UI resources bundled with package installation
+
+## Development Features
+
+### Hot Reloading (Beta)
+```python
+# Enable development mode
+app = LlamaIndexServer(
+    workflow_factory=create_workflow,
+    env="dev",                    # Required for dev features
+    ui_config={"dev_mode": True}, # Enable live editing
+)
+```
+- **Live Code Editing**: Modify workflow code in browser interface
+- **Automatic Restart**: FastAPI dev mode integration for instant updates
+- **File Watching**: Monitors `app/workflow.py` for changes
+
+### Logging and Debugging
+- **Verbose Mode**: Detailed request/response logging
+- **Error Handling**: Comprehensive exception catching and reporting
+- **Stream Monitoring**: Real-time event tracking during workflow execution
+
+## Integration Points
+
+### LlamaIndex Core
+- **Workflow Engine**: Full support for Workflow and AgentWorkflow classes
+- **Message Types**: Native ChatMessage and MessageRole compatibility
+- **Node Processing**: Automatic source node extraction and URL generation
+- **Tool Integration**: Function tools and external service connections
+
+### FastAPI Ecosystem
+- **Middleware**: CORS, authentication, and custom request processing
+- **Background Tasks**: Asynchronous file operations and processing
+- **Static Files**: Efficient serving of UI assets and generated content
+- **API Documentation**: Automatic OpenAPI/Swagger documentation generation
+
+### External Services
+- **LlamaCloud**: Cloud-based indexing and retrieval services
+- **File Readers**: Support for various document formats via LlamaIndex readers
+- **Code Interpreters**: Integration with E2B and other execution environments
+
+## Examples and Templates
+
+### Simple Workflow
+Basic agent with tool integration and starter questions for user guidance.
+
+### Agentic RAG
+Document retrieval system with vector indexing, query processing, and source citations.
+
+### Custom Layout
+Branded interface with custom header components and layout customization.
+
+### Development Mode
+Live code editing with hot reloading and separate workflow file organization.
+
+## Best Practices
+
+### Server Setup
+1. **Environment Variables**: Use `.env` files for API keys and configuration
+2. **Development vs Production**: Proper environment separation with `env` parameter
+3. **Resource Management**: Monitor memory usage with large document collections
+4. **Error Handling**: Implement comprehensive logging and exception handling
+
+### Workflow Design
+1. **Factory Pattern**: Use factory functions for stateless workflow creation
+2. **Event Emission**: Leverage `UIEvent` and `ArtifactEvent` for rich user experience
+3. **Message Handling**: Process chat history appropriately in workflow logic
+4. **Tool Integration**: Follow LlamaIndex patterns for external service connections
+
+### UI Development
+1. **Component Organization**: Structure custom components in dedicated directories
+2. **Event Schemas**: Design clear Pydantic models for UI generation
+3. **Layout Consistency**: Use shared layout components across workflows
+4. **Performance**: Consider event aggregation for large data sets
+
+This package provides a comprehensive foundation for deploying production-ready LlamaIndex applications with professional chat interfaces, extensible UI components, and robust API endpoints.

python/llama-index-server/README.md

@@ -8,6 +8,7 @@ LlamaIndexServer is a FastAPI-based application that allows you to quickly launc
 - Built on FastAPI for high performance and easy API development
 - Optional built-in chat UI with extendable UI components
 - Prebuilt development code
+- Human-in-the-loop (HITL) support, check out the [Human-in-the-loop](https://github.com/run-llama/create-llama/blob/main/python/llama-index-server/examples/hitl/README.md) documentation for more details.
 
 ## Installation
 

python/llama-index-server/examples/artifact/code_workflow.py

@@ -16,7 +16,8 @@ from llama_index.core.workflow import (
     Workflow,
     step,
 )
-from llama_index.server.api.models import (
+from llama_index.server.api.utils import get_last_artifact
+from llama_index.server.models import (
     Artifact,
     ArtifactEvent,
     ArtifactType,
@@ -24,7 +25,6 @@ from llama_index.server.api.models import (
     CodeArtifactData,
     UIEvent,
 )
-from llama_index.server.api.utils import get_last_artifact
 
 
 class Requirement(BaseModel):

python/llama-index-server/examples/artifact/document_workflow.py

@@ -16,7 +16,8 @@ from llama_index.core.workflow import (
     Workflow,
     step,
 )
-from llama_index.server.api.models import (
+from llama_index.server.api.utils import get_last_artifact
+from llama_index.server.models import (
     Artifact,
     ArtifactEvent,
     ArtifactType,
@@ -24,7 +25,6 @@ from llama_index.server.api.models import (
     DocumentArtifactData,
     UIEvent,
 )
-from llama_index.server.api.utils import get_last_artifact
 
 
 class DocumentRequirement(BaseModel):

python/llama-index-server/examples/artifact/layout/header.tsx

@@ -4,7 +4,7 @@ import { Sparkles, Star } from "lucide-react";
 
 export default function Header() {
   return (
-    <div className="flex items-center justify-between px-4 pt-2">
+    <div className="flex items-center justify-between p-2 px-4">
       <div className="flex items-center gap-2">
         <Sparkles className="size-4" />
         <h1 className="font-semibold">Artifact Workflow</h1>

python/llama-index-server/examples/artifact/main.py

@@ -7,7 +7,7 @@ from examples.artifact.code_workflow import ArtifactWorkflow
 from llama_index.core.workflow import Workflow
 from llama_index.llms.openai import OpenAI
 from llama_index.server import LlamaIndexServer, UIConfig
-from llama_index.server.api.models import ChatRequest
+from llama_index.server.models import ChatRequest
 
 
 def create_workflow(chat_request: ChatRequest) -> Workflow:

python/llama-index-server/examples/hitl/README.md

@@ -0,0 +1,74 @@
+# Human in the Loop
+
+This example shows how to use the LlamaIndexServer with a human in the loop.
+
+## AgentWorkflow
+
+```bash
+uv run -- agent_workflow.py
+```
+
+## Custom Workflow
+
+```bash
+uv run -- custom_workflow.py
+```
+
+## How does it work?
+The human-in-the-loop approach used here is based on a simple idea: the workflow pauses and waits for a human response before proceeding to the next step.
+
+To do this, you will need to implement two custom events: 
++ [HumanInputEvent](../../llama_index/server/models/hitl.py#L10): This event is used to request input from the user.
++ [HumanResponseEvent](../../llama_index/server/models/hitl.py#L43): This event is sent to the workflow to resume execution with input from the user.
+
+In this example, we have implemented these two custom events:  
+
+- [CLIHumanInputEvent](events.py#L20) – to request input from the user for CLI command execution.
+- [CLIHumanResponseEvent](events.py#L8) – to resume the workflow with the response from the user.
+
+We also have a custom component, [cli_human_input.tsx](./components/cli_human_input.tsx), which displays a card that the user can update the command and choose to execute or cancel the command execution.
+
+To make the [AgentWorkflow](agent_workflow.py) work, we use the `wait_for_event()` method to wait for the human response when a tool is called.
+
+Example:
+```python
+async def cli_executor(ctx: Context, command: str) -> str:
+    """
+    This tool carefully waits for user confirmation before executing a command.
+    """
+    confirmation = await ctx.wait_for_event(
+        CLIHumanResponseEvent,
+        waiter_event=CLIHumanInputEvent(
+            data=CLICommand(command=command),
+        ),
+    )
+    if confirmation.execute:
+        # Execute the command
+        ...
+    else:
+        # Cancel the command
+        ...
+
+```
+
+And for [Custom Workflow](custom_workflow.py), we can define a step that send the `CLIHumanInputEvent` and another step that wait for the `CLIHumanResponseEvent`.
+
+Example:
+```python
+@step
+async def request_input(self, ctx: Context, ev: StartEvent) -> CLIHumanInputEvent:
+    ...
+    return CLIHumanInputEvent(
+        data=CLICommand(command=command),
+        response_event_type=CLIHumanResponseEvent,
+    )
+
+@step
+async def handle_human_response(self, ctx: Context, ev: CLIHumanResponseEvent) -> StopEvent:
+    if ev.execute:
+        # Execute the command
+        ...
+    else:
+        # Cancel the command
+        ...
+```

python/llama-index-server/examples/hitl/agent_workflow.py

@@ -0,0 +1,60 @@
+import subprocess
+
+from events import CLICommand, CLIHumanInputEvent, CLIHumanResponseEvent
+from fastapi import FastAPI
+
+from llama_index.core.agent.workflow import AgentWorkflow
+from llama_index.core.workflow import Context
+from llama_index.llms.openai import OpenAI
+from llama_index.server import LlamaIndexServer, UIConfig
+
+
+async def cli_executor(ctx: Context, command: str) -> str:
+    """
+    This tool carefully waits for user confirmation before executing a command.
+    """
+    confirmation = await ctx.wait_for_event(
+        CLIHumanResponseEvent,
+        waiter_event=CLIHumanInputEvent(
+            data=CLICommand(command=command),
+        ),
+    )
+    if confirmation.execute:
+        return subprocess.check_output(confirmation.command, shell=True).decode("utf-8")
+    else:
+        return "Command execution cancelled."
+
+
+def create_workflow() -> AgentWorkflow:
+    return AgentWorkflow.from_tools_or_functions(
+        tools_or_functions=[cli_executor],
+        llm=OpenAI(model="gpt-4.1-mini"),
+        system_prompt="""
+        You are a helpful assistant that help the user execute commands.
+        You can execute commands using the cli_executor tool, don't need to ask for confirmation for triggering the tool.
+        """,
+    )
+
+
+def create_app() -> FastAPI:
+    app = LlamaIndexServer(
+        workflow_factory=create_workflow,
+        suggest_next_questions=False,
+        ui_config=UIConfig(
+            starter_questions=[
+                "List all files in the current directory",
+                "Fetch changes from the remote repository",
+            ],
+            component_dir="components",
+        ),
+    )
+    return app
+
+
+app = create_app()
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run("agent_workflow:app", port=8000, reload=True)

python/llama-index-server/examples/hitl/components/cli_human_input.tsx

@@ -0,0 +1,96 @@
+import { JSONValue, useChatUI } from "@llamaindex/chat-ui";
+import React, { FC, useState } from "react";
+import { Button } from "@/components/ui/button";
+import { Card, CardContent, CardFooter } from "@/components/ui/card";
+import { z } from "zod";
+
+// This schema is equivalent to the CLICommand model defined in events.py
+const CLIInputEventSchema = z.object({
+  command: z.string(),
+});
+type CLIInputEvent = z.infer<typeof CLIInputEventSchema>;
+
+
+const CLIHumanInput: FC<{
+  events: JSONValue[];
+}> = ({ events }) => {
+  const inputEvent = (events || [])
+    .map((ev) => {
+      const parseResult = CLIInputEventSchema.safeParse(ev);
+      return parseResult.success ? parseResult.data : null;
+    })
+    .filter((ev): ev is CLIInputEvent => ev !== null)
+    .at(-1);
+
+  const { append } = useChatUI();
+  const [confirmedValue, setConfirmedValue] = useState<boolean | null>(null);
+  const [editableCommand, setEditableCommand] = useState<string | undefined>(
+    inputEvent?.command,
+  );
+
+  // Update editableCommand if inputEvent changes (e.g. new event comes in)
+  React.useEffect(() => {
+    setEditableCommand(inputEvent?.command);
+  }, [inputEvent?.command]);
+
+  const handleConfirm = () => {
+    append({
+      content: "Yes",
+      role: "user",
+      annotations: [
+        {
+          type: "human_response",
+          data: {
+            execute: true,
+            command: editableCommand, // Use editable command
+          },
+        },
+      ],
+    });
+    setConfirmedValue(true);
+  };
+
+  const handleCancel = () => {
+    append({
+      content: "No",
+      role: "user",
+      annotations: [
+        {
+          type: "human_response",
+          data: {
+            execute: false,
+            command: inputEvent?.command,
+          },
+        },
+      ],
+    });
+    setConfirmedValue(false);
+  };
+
+  return (
+    <Card className="my-4">
+      <CardContent className="pt-6">
+        <p className="text-sm text-gray-700">
+          Do you want to execute the following command?
+        </p>
+        <input
+          disabled
+          type="text"
+          value={editableCommand || ""}
+          onChange={(e) => setEditableCommand(e.target.value)}
+          className="bg-gray-100 rounded p-3 my-2 text-xs font-mono text-gray-800 overflow-x-auto w-full border border-gray-300"
+        />
+      </CardContent>
+      {confirmedValue === null ? (
+        <CardFooter className="flex justify-end gap-2">
+          <>
+            <Button onClick={handleConfirm}>Yes</Button>
+            <Button onClick={handleCancel}>No</Button>
+          </>
+        </CardFooter>
+      ) : null}
+    </Card>
+  );
+};
+
+export default CLIHumanInput;

python/llama-index-server/examples/hitl/custom_workflow.py

@@ -0,0 +1,109 @@
+import platform
+import subprocess
+from typing import Any
+
+from events import CLICommand, CLIHumanInputEvent, CLIHumanResponseEvent
+from fastapi import FastAPI
+
+from llama_index.core.prompts import PromptTemplate
+from llama_index.core.settings import Settings
+from llama_index.core.workflow import (
+    Context,
+    StartEvent,
+    StopEvent,
+    Workflow,
+    step,
+)
+from llama_index.server import LlamaIndexServer, UIConfig
+
+
+class CLIWorkflow(Workflow):
+    """
+    A workflow has ability to execute command line tool with human in the loop for confirmation.
+    """
+
+    default_prompt = PromptTemplate(
+        template="""
+        You are a helpful assistant who can write CLI commands to execute using {cli_language}.
+        Your task is to analyze the user's request and write a CLI command to execute.
+
+        ## User Request
+        {user_request}
+
+        Don't be verbose, only respond with the CLI command without any other text.
+        """
+    )
+
+    def __init__(self, **kwargs: Any) -> None:
+        # HITL Workflow should disable timeout otherwise, we will get a timeout error from callback
+        kwargs["timeout"] = None
+        super().__init__(**kwargs)
+
+    @step
+    async def start(self, ctx: Context, ev: StartEvent) -> CLIHumanInputEvent:
+        user_msg = ev.user_msg
+        if user_msg is None:
+            raise ValueError("Missing user_msg in StartEvent")
+        await ctx.set("user_msg", user_msg)
+        # Request LLM to generate a CLI command
+        os_name = platform.system()
+        if os_name == "Linux" or os_name == "Darwin":
+            cli_language = "bash"
+        else:
+            cli_language = "cmd"
+        prompt = self.default_prompt.format(
+            user_request=user_msg, cli_language=cli_language
+        )
+        llm = Settings.llm
+        if llm is None:
+            raise ValueError("Missing LLM in Settings")
+        response = await llm.acomplete(prompt, formatted=True)
+        command = response.text.strip()
+        if command == "":
+            raise ValueError("Couldn't generate a command")
+        # Send the command to the user for confirmation
+        await ctx.set("command", command)
+        return CLIHumanInputEvent(  # type: ignore
+            data=CLICommand(command=command),
+            response_event_type=CLIHumanResponseEvent,
+        )
+
+    @step
+    async def handle_human_response(
+        self,
+        ctx: Context,
+        ev: CLIHumanResponseEvent,  # This event is sent by LlamaIndexServer when user response
+    ) -> StopEvent:
+        # If we have human response, check the confirmation and execute the command
+        if ev.execute:
+            command = ev.command or ""
+            if command == "":
+                raise ValueError("Missing command in CLIExecutionEvent")
+            res = subprocess.run(command, shell=True, capture_output=True, text=True)
+            return StopEvent(result=res.stdout or res.stderr)
+        else:
+            return StopEvent(result=None)
+
+
+def create_app() -> FastAPI:
+    app = LlamaIndexServer(
+        workflow_factory=lambda: CLIWorkflow(),
+        suggest_next_questions=False,
+        ui_config=UIConfig(
+            starter_questions=[
+                "List all files in the current directory",
+                "Fetch changes from the remote repository",
+            ],
+            component_dir="components",
+        ),
+    )
+    return app
+
+
+app = create_app()
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run("custom_workflow:app", port=8000, reload=True)

python/llama-index-server/examples/hitl/events.py

@@ -0,0 +1,34 @@
+from typing import Type
+
+from pydantic import BaseModel, Field
+
+from llama_index.server.models import HumanInputEvent, HumanResponseEvent
+
+
+class CLIHumanResponseEvent(HumanResponseEvent):
+    execute: bool = Field(
+        description="True if the human wants to execute the command, False otherwise."
+    )
+    command: str = Field(description="The command to execute.")
+
+
+class CLICommand(BaseModel):
+    command: str = Field(description="The command to execute.")
+
+
+# We need an event that extends from HumanInputEvent for HITL feature
+class CLIHumanInputEvent(HumanInputEvent):
+    """
+    CLIInputRequiredEvent is sent when the agent needs permission from the user to execute the CLI command or not.
+    Render this event by showing the command and a boolean button to execute the command or not.
+    """
+
+    event_type: str = (
+        "cli_human_input"  # used by UI to render with appropriate component
+    )
+    response_event_type: Type = (
+        CLIHumanResponseEvent  # used by workflow to resume with the correct event
+    )
+    data: CLICommand = Field(  # the data that sent to the UI for rendering
+        description="The command to execute.",
+    )

python/llama-index-server/examples/llamacloud/main.py

@@ -0,0 +1,114 @@
+import os
+from typing import List, Optional
+
+from fastapi import FastAPI
+
+from llama_index.core.agent.workflow import AgentWorkflow
+from llama_index.core.query_engine.retriever_query_engine import RetrieverQueryEngine
+from llama_index.core.settings import Settings
+from llama_index.core.tools import QueryEngineTool, ToolMetadata
+from llama_index.llms.openai import OpenAI
+from llama_index.server import LlamaIndexServer, UIConfig
+from llama_index.server.models import ChatRequest
+from llama_index.server.services.llamacloud import LlamaCloudIndex, get_index
+from llama_index.server.tools.index.citation import (
+    CITATION_SYSTEM_PROMPT,
+    enable_citation,
+)
+
+# Please set the following environment variables to use LlamaCloud
+if os.getenv("LLAMA_CLOUD_API_KEY") is None:
+    raise ValueError("LLAMA_CLOUD_API_KEY is not set")
+if os.getenv("LLAMA_CLOUD_PROJECT_NAME") is None:
+    raise ValueError("LLAMA_CLOUD_PROJECT_NAME is not set")
+if os.getenv("LLAMA_CLOUD_INDEX_NAME") is None:
+    raise ValueError("LLAMA_CLOUD_INDEX_NAME is not set")
+
+Settings.llm = OpenAI(model="gpt-4.1")
+
+
+def get_tools(index: LlamaCloudIndex) -> List[QueryEngineTool]:
+    """
+    Get the tools for the given index.
+    """
+
+    chunk_retriever = index.as_retriever(
+        retrieval_mode="chunks",
+        rerank_top_n=15,
+        dense_similarity_top_k=1,
+    )
+    doc_retriever = index.as_retriever(
+        retrieval_mode="files_via_content",
+        files_top_k=1,
+    )
+
+    # You can either create query engine with CitationSynthesizer and NodeCitationProcessor
+    # or use the enable_citation function to enable citation for the query engine.
+    chunk_engine = RetrieverQueryEngine.from_args(
+        retriever=chunk_retriever,
+        llm=Settings.llm,
+    )
+    doc_engine = RetrieverQueryEngine.from_args(
+        retriever=doc_retriever,
+        llm=Settings.llm,
+    )
+
+    chunk_tool = QueryEngineTool(
+        query_engine=chunk_engine,
+        metadata=ToolMetadata(
+            name="chunk_query_engine",
+            description=(
+                "Get answer from specific chunk of a given document. Best used for lower-level questions that require specific information from a given document."
+                "Do NOT use if the answer can be found in the entire document. Use the file_query_engine instead for that purpose"
+            ),
+        ),
+    )
+    doc_tool = QueryEngineTool(
+        query_engine=doc_engine,
+        metadata=ToolMetadata(
+            name="file_query_engine",
+            description=(
+                "Get answer from entire document as context.  Best used for higher-level summarization questions."
+                "Do NOT use if the answer can be found in a specific chunk of a given document. Use the chunk_query_engine instead for that purpose"
+            ),
+        ),
+    )
+
+    return [enable_citation(chunk_tool), enable_citation(doc_tool)]
+
+
+def create_workflow(chat_request: Optional[ChatRequest] = None) -> AgentWorkflow:
+    index = get_index(chat_request=chat_request)
+    if index is None:
+        raise RuntimeError("Index not found!")
+
+    # Append the citation system prompt to the system prompt
+    system_prompt = """
+    You are a helpful assistant that has access to a knowledge base.
+    """
+    system_prompt += CITATION_SYSTEM_PROMPT
+    return AgentWorkflow.from_tools_or_functions(
+        tools_or_functions=get_tools(index),
+        system_prompt=system_prompt,
+    )
+
+
+def create_app() -> FastAPI:
+    app = LlamaIndexServer(
+        workflow_factory=create_workflow,
+        env="dev",
+        suggest_next_questions=False,
+        ui_config=UIConfig(
+            llamacloud_index_selector=True,  # to select different indexes in the UI
+        ),
+    )
+    return app
+
+
+app = create_app()
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

python/llama-index-server/examples/simple-agent/app/workflow.py

@@ -3,7 +3,7 @@ from typing import Optional
 from llama_index.core.agent.workflow import AgentWorkflow
 from llama_index.core.settings import Settings
 from llama_index.llms.openai import OpenAI
-from llama_index.server.api.models import ChatRequest
+from llama_index.server.models import ChatRequest
 
 
 def create_workflow(chat_request: Optional[ChatRequest] = None) -> AgentWorkflow:

python/llama-index-server/llama_index/server/__init__.py

@@ -1,4 +1,4 @@
-from .api.models import UIEvent
+from .models.ui import UIEvent
 from .server import LlamaIndexServer, UIConfig
 
 __all__ = ["LlamaIndexServer", "UIConfig", "UIEvent"]

python/llama-index-server/llama_index/server/api/callbacks/__init__.py

@@ -1,3 +1,4 @@
+from llama_index.server.api.callbacks.agent_call_tool import AgentCallTool
 from llama_index.server.api.callbacks.base import EventCallback
 from llama_index.server.api.callbacks.llamacloud import LlamaCloudFileDownload
 from llama_index.server.api.callbacks.source_nodes import SourceNodesFromToolCall
@@ -10,4 +11,5 @@ __all__ = [
     "SourceNodesFromToolCall",
     "SuggestNextQuestions",
     "LlamaCloudFileDownload",
+    "AgentCallTool",
 ]

python/llama-index-server/llama_index/server/api/callbacks/agent_call_tool.py

@@ -0,0 +1,26 @@
+import logging
+from typing import Any
+
+from llama_index.core.agent.workflow.workflow_events import ToolCall, ToolCallResult
+from llama_index.server.api.callbacks.base import EventCallback
+from llama_index.server.models.ui import AgentRunEvent
+
+logger = logging.getLogger("uvicorn")
+
+
+class AgentCallTool(EventCallback):
+    """
+    Adapter for convert tool call events to agent run events.
+    """
+
+    async def run(self, event: Any) -> Any:
+        if isinstance(event, ToolCall) and not isinstance(event, ToolCallResult):
+            return AgentRunEvent(
+                name="Agent",
+                msg=f"Calling tool: {event.tool_name} with: {event.tool_kwargs}",
+            )
+        return event
+
+    @classmethod
+    def from_default(cls, *args: Any, **kwargs: Any) -> "AgentCallTool":
+        return cls()

python/llama-index-server/llama_index/server/api/callbacks/source_nodes.py

@@ -1,31 +1,51 @@
-from typing import Any
+import logging
+from typing import Any, List, Optional
 
 from llama_index.core.agent.workflow.workflow_events import ToolCallResult
+from llama_index.core.schema import NodeWithScore
 from llama_index.server.api.callbacks.base import EventCallback
-from llama_index.server.api.models import SourceNodesEvent
+from llama_index.server.models.source_nodes import SourceNodesEvent
+
+logger = logging.getLogger(__name__)
 
 
 class SourceNodesFromToolCall(EventCallback):
     """
     Extract source nodes from the query tool output.
-
-    Args:
-        query_tool_name: The name of the tool that queries the index.
-                         default is "query_index"
     """
 
-    def __init__(self, query_tool_name: str = "query_index"):
-        self.query_tool_name = query_tool_name
+    def __init__(self, tool_name: Optional[str] = None):
+        # backward compatibility
+        if tool_name is not None:
+            logger.warning(
+                "tool_name has been deprecated. It's now detected by the tool output."
+            )
 
-    def transform_tool_call_result(self, event: ToolCallResult) -> SourceNodesEvent:
-        source_nodes = event.tool_output.raw_output.source_nodes
-        return SourceNodesEvent(nodes=source_nodes)
+    def _get_source_nodes(self, event: ToolCallResult) -> Optional[List[NodeWithScore]]:
+        # If result is not error
+        if event.tool_output.is_error:
+            return None
+        # If result is not error, check if source nodes are in the tool output
+        raw_output = event.tool_output.raw_output
+        if hasattr(raw_output, "source_nodes"):
+            source_nodes = raw_output.source_nodes
+            # Verify if source_nodes is List[NodeWithScore]
+            if isinstance(source_nodes, list) and all(
+                isinstance(node, NodeWithScore) for node in source_nodes
+            ):
+                return source_nodes
+            else:
+                return None
+        else:
+            return None
 
     async def run(self, event: Any) -> Any:
+        events = [event]
         if isinstance(event, ToolCallResult):
-            if event.tool_name == self.query_tool_name:
-                return event, self.transform_tool_call_result(event)
-        return event
+            source_nodes = self._get_source_nodes(event)
+            if source_nodes is not None:
+                events.append(SourceNodesEvent(nodes=source_nodes))
+        return events
 
     @classmethod
     def from_default(cls, *args: Any, **kwargs: Any) -> "SourceNodesFromToolCall":

python/llama-index-server/llama_index/server/api/callbacks/suggest_next_questions.py

@@ -2,7 +2,7 @@ import logging
 from typing import Any, Optional
 
 from llama_index.server.api.callbacks.base import EventCallback
-from llama_index.server.api.models import ChatRequest
+from llama_index.server.models.chat import ChatRequest
 from llama_index.server.services.suggest_next_question import (
     SuggestNextQuestionsService,
 )

python/llama-index-server/llama_index/server/api/models.py

@@ -1,198 +1,2 @@
-import logging
-import os
-from enum import Enum
-from typing import Any, Dict, List, Literal, Optional, Union
-
-from pydantic import BaseModel, field_validator
-
-from llama_index.core.schema import NodeWithScore
-from llama_index.core.types import ChatMessage, MessageRole
-from llama_index.core.workflow import Event
-from llama_index.server.settings import server_settings
-
-logger = logging.getLogger("uvicorn")
-
-
-class ChatAPIMessage(BaseModel):
-    role: MessageRole
-    content: str
-    annotations: Optional[List[Any]] = None
-
-    def to_llamaindex_message(self) -> ChatMessage:
-        return ChatMessage(role=self.role, content=self.content)
-
-
-class ChatRequest(BaseModel):
-    messages: List[ChatAPIMessage]
-    data: Optional[Any] = None
-
-    @field_validator("messages")
-    def validate_messages(cls, v: List[ChatAPIMessage]) -> List[ChatAPIMessage]:
-        if v[-1].role != MessageRole.USER:
-            raise ValueError("Last message must be from user")
-        return v
-
-
-class AgentRunEventType(Enum):
-    TEXT = "text"
-    PROGRESS = "progress"
-
-
-class AgentRunEvent(Event):
-    name: str
-    msg: str
-    event_type: AgentRunEventType = AgentRunEventType.TEXT
-    data: Optional[dict] = None
-
-    def to_response(self) -> dict:
-        return {
-            "type": "agent",
-            "data": {
-                "agent": self.name,
-                "type": self.event_type.value,
-                "text": self.msg,
-                "data": self.data,
-            },
-        }
-
-
-class SourceNodesEvent(Event):
-    nodes: List[NodeWithScore]
-
-    def to_response(self) -> dict:
-        return {
-            "type": "sources",
-            "data": {
-                "nodes": [
-                    SourceNodes.from_source_node(node).model_dump()
-                    for node in self.nodes
-                ]
-            },
-        }
-
-
-class SourceNodes(BaseModel):
-    id: str
-    metadata: Dict[str, Any]
-    score: Optional[float]
-    text: str
-    url: Optional[str]
-
-    @classmethod
-    def from_source_node(cls, source_node: NodeWithScore) -> "SourceNodes":
-        metadata = source_node.node.metadata
-        url = cls.get_url_from_metadata(metadata)
-
-        return cls(
-            id=source_node.node.node_id,
-            metadata=metadata,
-            score=source_node.score,
-            text=source_node.node.text,  # type: ignore
-            url=url,
-        )
-
-    @classmethod
-    def get_url_from_metadata(
-        cls,
-        metadata: Dict[str, Any],
-        data_dir: Optional[str] = None,
-    ) -> Optional[str]:
-        url_prefix = server_settings.file_server_url_prefix
-        if data_dir is None:
-            data_dir = "data"
-        file_name = metadata.get("file_name")
-
-        if file_name and url_prefix:
-            # file_name exists and file server is configured
-            pipeline_id = metadata.get("pipeline_id")
-            if pipeline_id:
-                # file is from LlamaCloud
-                file_name = f"{pipeline_id}${file_name}"
-                return f"{url_prefix}/output/llamacloud/{file_name}"
-            is_private = metadata.get("private", "false") == "true"
-            if is_private:
-                # file is a private upload
-                return f"{url_prefix}/output/uploaded/{file_name}"
-            # file is from calling the 'generate' script
-            # Get the relative path of file_path to data_dir
-            file_path = metadata.get("file_path")
-            data_dir = os.path.abspath(data_dir)
-            if file_path and data_dir:
-                relative_path = os.path.relpath(file_path, data_dir)
-                return f"{url_prefix}/data/{relative_path}"
-        # fallback to URL in metadata (e.g. for websites)
-        return metadata.get("URL")
-
-    @classmethod
-    def from_source_nodes(
-        cls, source_nodes: List[NodeWithScore]
-    ) -> List["SourceNodes"]:
-        return [cls.from_source_node(node) for node in source_nodes]
-
-
-class ComponentDefinition(BaseModel):
-    type: str
-    code: str
-    filename: str
-
-
-class UIEvent(Event):
-    type: str
-    data: BaseModel
-
-    def to_response(self) -> dict:
-        return {
-            "type": self.type,
-            "data": self.data.model_dump(),
-        }
-
-
-class ArtifactType(str, Enum):
-    CODE = "code"
-    DOCUMENT = "document"
-
-
-class CodeArtifactData(BaseModel):
-    file_name: str
-    code: str
-    language: str
-
-
-class DocumentArtifactData(BaseModel):
-    title: str
-    content: str
-    type: Literal["markdown", "html"]
-
-
-class Artifact(BaseModel):
-    created_at: Optional[int] = None
-    type: ArtifactType
-    data: Union[CodeArtifactData, DocumentArtifactData]
-
-    @classmethod
-    def from_message(cls, message: ChatAPIMessage) -> Optional["Artifact"]:
-        if not message.annotations or not isinstance(message.annotations, list):
-            return None
-
-        for annotation in message.annotations:
-            if isinstance(annotation, dict) and annotation.get("type") == "artifact":
-                try:
-                    artifact = cls.model_validate(annotation.get("data"))
-                    return artifact
-                except Exception as e:
-                    logger.warning(
-                        f"Failed to parse artifact from annotation: {annotation}. Error: {e}"
-                    )
-
-        return None
-
-
-class ArtifactEvent(Event):
-    type: str = "artifact"
-    data: Artifact
-
-    def to_response(self) -> dict:
-        return {
-            "type": self.type,
-            "data": self.data.model_dump(),
-        }
+# TODO: For backward compatibility, remove this in a minor release
+from llama_index.server.models import *  # noqa

python/llama-index-server/llama_index/server/api/routers/chat.py

@@ -6,23 +6,28 @@ from typing import AsyncGenerator, Callable, Union
 
 from fastapi import APIRouter, BackgroundTasks, HTTPException
 from fastapi.responses import StreamingResponse
-
 from llama_index.core.agent.workflow.workflow_events import (
     AgentInput,
     AgentSetup,
     AgentStream,
 )
-from llama_index.core.workflow import StopEvent, Workflow
+from llama_index.core.workflow import (
+    StopEvent,
+    Workflow,
+)
 from llama_index.server.api.callbacks import (
+    AgentCallTool,
     EventCallback,
     LlamaCloudFileDownload,
     SourceNodesFromToolCall,
     SuggestNextQuestions,
 )
 from llama_index.server.api.callbacks.stream_handler import StreamHandler
-from llama_index.server.api.models import ChatRequest
 from llama_index.server.api.utils.vercel_stream import VercelStreamResponse
+from llama_index.server.models.chat import ChatRequest
+from llama_index.server.models.hitl import HumanInputEvent
 from llama_index.server.services.llamacloud import LlamaCloudFileService
+from llama_index.server.services.workflow import HITLWorkflowService
 
 
 def chat_router(
@@ -38,7 +43,8 @@ def chat_router(
         background_tasks: BackgroundTasks,
     ) -> StreamingResponse:
         try:
-            user_message = request.messages[-1].to_llamaindex_message()
+            last_message = request.messages[-1]
+            user_message = last_message.to_llamaindex_message()
             chat_history = [
                 message.to_llamaindex_message() for message in request.messages[:-1]
             ]
@@ -48,12 +54,24 @@ def chat_router(
                 workflow = workflow_factory(chat_request=request)
             else:
                 workflow = workflow_factory()
-            workflow_handler = workflow.run(
-                user_msg=user_message.content,
-                chat_history=chat_history,
-            )
+
+            # Check if we should resume a chat with a human response
+            human_response = last_message.human_response
+            if human_response:
+                ctx = await HITLWorkflowService.load_context(
+                    id=request.id,
+                    workflow=workflow,
+                    data=human_response,
+                )
+                workflow_handler = workflow.run(ctx=ctx)
+            else:
+                workflow_handler = workflow.run(
+                    user_msg=user_message.content,
+                    chat_history=chat_history,
+                )
 
             callbacks: list[EventCallback] = [
+                AgentCallTool(),
                 SourceNodesFromToolCall(),
                 LlamaCloudFileDownload(background_tasks),
             ]
@@ -65,7 +83,11 @@ def chat_router(
             )
 
             return VercelStreamResponse(
-                content_generator=_stream_content(stream_handler, request, logger),
+                content_generator=_stream_content(
+                    stream_handler,
+                    logger,
+                    request.id,
+                ),
             )
         except Exception as e:
             logger.error(e)
@@ -98,8 +120,8 @@ def chat_router(
 
 async def _stream_content(
     handler: StreamHandler,
-    request: ChatRequest,
     logger: logging.Logger,
+    chat_id: str,
 ) -> AsyncGenerator[str, None]:
     async def _text_stream(
         event: Union[AgentStream, StopEvent],
@@ -125,6 +147,19 @@ async def _stream_content(
                 async for chunk in _text_stream(event):
                     handler.accumulate_text(chunk)
                     yield VercelStreamResponse.convert_text(chunk)
+            elif isinstance(event, HumanInputEvent):
+                ctx = handler.workflow_handler.ctx
+                if ctx is None:
+                    raise RuntimeError("Context is None")
+                # Save the context with the HITL event
+                await HITLWorkflowService.save_context(
+                    id=chat_id,
+                    ctx=ctx,
+                    resume_event_type=event.response_event_type,
+                )
+                yield VercelStreamResponse.convert_data(event.to_response())
+                # Break to stop the stream
+                break
             elif isinstance(event, dict):
                 yield VercelStreamResponse.convert_data(event)
             elif hasattr(event, "to_response"):

python/llama-index-server/llama_index/server/api/routers/ui.py

@@ -2,7 +2,8 @@ import logging
 from typing import List
 
 from fastapi import APIRouter
-from llama_index.server.api.models import ComponentDefinition
+
+from llama_index.server.models.ui import ComponentDefinition
 from llama_index.server.services.custom_ui import CustomUI
 
 

python/llama-index-server/llama_index/server/api/utils/chat_request.py

@@ -1,6 +1,7 @@
 from typing import List, Optional
 
-from llama_index.server.api.models import Artifact, ChatRequest
+from llama_index.server.models.artifacts import Artifact
+from llama_index.server.models.chat import ChatRequest
 
 
 def get_artifacts(chat_request: ChatRequest) -> List[Artifact]:

python/llama-index-server/llama_index/server/models/__init__.py

@@ -0,0 +1,34 @@
+from llama_index.server.models.artifacts import (
+    Artifact,
+    ArtifactEvent,
+    ArtifactType,
+    CodeArtifactData,
+    DocumentArtifactData,
+)
+from llama_index.server.models.chat import ChatAPIMessage, ChatRequest
+from llama_index.server.models.hitl import HumanInputEvent, HumanResponseEvent
+from llama_index.server.models.source_nodes import SourceNodes, SourceNodesEvent
+from llama_index.server.models.ui import (
+    AgentRunEvent,
+    AgentRunEventType,
+    ComponentDefinition,
+    UIEvent,
+)
+
+__all__ = [
+    "Artifact",
+    "ArtifactEvent",
+    "ArtifactType",
+    "DocumentArtifactData",
+    "CodeArtifactData",
+    "ChatAPIMessage",
+    "ChatRequest",
+    "UIEvent",
+    "ComponentDefinition",
+    "AgentRunEvent",
+    "AgentRunEventType",
+    "SourceNodes",
+    "SourceNodesEvent",
+    "HumanInputEvent",
+    "HumanResponseEvent",
+]

python/llama-index-server/llama_index/server/models/artifacts.py

@@ -0,0 +1,60 @@
+import logging
+from enum import Enum
+from typing import Literal, Optional, Union
+
+from llama_index.core.workflow.events import Event
+from llama_index.server.models.chat import ChatAPIMessage
+from pydantic import BaseModel
+
+logger = logging.getLogger(__name__)
+
+
+class ArtifactType(str, Enum):
+    CODE = "code"
+    DOCUMENT = "document"
+
+
+class CodeArtifactData(BaseModel):
+    file_name: str
+    code: str
+    language: str
+
+
+class DocumentArtifactData(BaseModel):
+    title: str
+    content: str
+    type: Literal["markdown", "html"]
+
+
+class Artifact(BaseModel):
+    created_at: Optional[int] = None
+    type: ArtifactType
+    data: Union[CodeArtifactData, DocumentArtifactData]
+
+    @classmethod
+    def from_message(cls, message: ChatAPIMessage) -> Optional["Artifact"]:
+        if not message.annotations or not isinstance(message.annotations, list):
+            return None
+
+        for annotation in message.annotations:
+            if isinstance(annotation, dict) and annotation.get("type") == "artifact":
+                try:
+                    artifact = cls.model_validate(annotation.get("data"))
+                    return artifact
+                except Exception as e:
+                    logger.warning(
+                        f"Failed to parse artifact from annotation: {annotation}. Error: {e}"
+                    )
+
+        return None
+
+
+class ArtifactEvent(Event):
+    type: str = "artifact"
+    data: Artifact
+
+    def to_response(self) -> dict:
+        return {
+            "type": self.type,
+            "data": self.data.model_dump(),
+        }

python/llama-index-server/llama_index/server/models/chat.py

@@ -0,0 +1,44 @@
+import re
+from typing import Any, List, Optional
+
+from pydantic import BaseModel, field_validator
+
+from llama_index.core.types import ChatMessage, MessageRole
+
+
+class ChatAPIMessage(BaseModel):
+    role: MessageRole
+    content: str
+    annotations: Optional[List[Any]] = None
+
+    def to_llamaindex_message(self) -> ChatMessage:
+        return ChatMessage(role=self.role, content=self.content)
+
+    @property
+    def human_response(self) -> Optional[Any]:
+        if self.annotations:
+            for annotation in self.annotations:
+                if (
+                    isinstance(annotation, dict)
+                    and annotation.get("type") == "human_response"
+                ):
+                    return annotation.get("data", {})
+        return None
+
+
+class ChatRequest(BaseModel):
+    id: str  # see https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat#id - constant for the same chat session
+    messages: List[ChatAPIMessage]
+    data: Optional[Any] = None
+
+    @field_validator("messages")
+    def validate_messages(cls, v: List[ChatAPIMessage]) -> List[ChatAPIMessage]:
+        if v[-1].role != MessageRole.USER:
+            raise ValueError("Last message must be from user")
+        return v
+
+    @field_validator("id")
+    def validate_id(cls, v: str) -> str:
+        if re.search(r"[^a-zA-Z0-9_-]", v):
+            raise ValueError("ID contains special characters")
+        return v

python/llama-index-server/llama_index/server/models/hitl.py

@@ -0,0 +1,51 @@
+from typing import Any, Dict, Type, Union
+
+from llama_index.core.workflow.events import (
+    HumanResponseEvent as FrameworkHumanResponseEvent,
+)
+from llama_index.core.workflow.events import InputRequiredEvent
+from pydantic import BaseModel, Field
+
+
+class HumanResponseEvent(FrameworkHumanResponseEvent):
+    """
+    Use this event to send a response from a human.
+    """
+
+    def __init__(self, **kwargs: Any) -> None:
+        if "response" not in kwargs:
+            kwargs["response"] = f"Human response with data: {kwargs.get('data', {})}"
+        super().__init__(**kwargs)
+
+
+class HumanInputEvent(InputRequiredEvent):
+    """
+    Use this event to request input from a human.
+    It will block the workflow execution until the human responds.
+    """
+
+    response_event_type: Type[HumanResponseEvent] = Field(
+        description="The type of event that the workflow is waiting for.",
+    )
+    event_type: str = Field(
+        description="An identifier for the UI component that will be used to render the input.",
+    )
+    data: Union[Dict[str, Any], BaseModel] = Field(
+        description="The data to be sent to the UI component that will be used to render the input.",
+    )
+
+    def __init__(self, **kwargs: Any) -> None:
+        # Construct the prefix for InputRequiredEvent
+        event_type = kwargs.get("event_type", None)
+        data = kwargs.get("data", None)
+        if "prefix" not in kwargs:
+            kwargs["prefix"] = f"Need input for {event_type} with data: {data}"
+        super().__init__(**kwargs)
+
+    def to_response(self) -> dict:
+        return {
+            "type": self.event_type,
+            "data": self.data
+            if isinstance(self.data, dict)
+            else self.data.model_dump(),
+        }

python/llama-index-server/llama_index/server/models/source_nodes.py

@@ -0,0 +1,49 @@
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel
+
+from llama_index.core.schema import NodeWithScore
+from llama_index.core.workflow.events import Event
+from llama_index.server.utils.chat_file import get_file_url_from_metadata
+
+
+class SourceNodesEvent(Event):
+    nodes: List[NodeWithScore]
+
+    def to_response(self) -> dict:
+        return {
+            "type": "sources",
+            "data": {
+                "nodes": [
+                    SourceNodes.from_source_node(node).model_dump()
+                    for node in self.nodes
+                ]
+            },
+        }
+
+
+class SourceNodes(BaseModel):
+    id: str
+    metadata: Dict[str, Any]
+    score: Optional[float]
+    text: str
+    url: Optional[str]
+
+    @classmethod
+    def from_source_node(cls, source_node: NodeWithScore) -> "SourceNodes":
+        metadata = source_node.node.metadata
+        url = get_file_url_from_metadata(metadata)
+
+        return cls(
+            id=source_node.node.node_id,
+            metadata=metadata,
+            score=source_node.score,
+            text=source_node.node.text,  # type: ignore
+            url=url,
+        )
+
+    @classmethod
+    def from_source_nodes(
+        cls, source_nodes: List[NodeWithScore]
+    ) -> List["SourceNodes"]:
+        return [cls.from_source_node(node) for node in source_nodes]

python/llama-index-server/llama_index/server/models/ui.py

@@ -0,0 +1,49 @@
+import logging
+from enum import Enum
+from typing import Optional
+
+from pydantic import BaseModel
+
+from llama_index.core.workflow import Event
+
+logger = logging.getLogger("uvicorn")
+
+
+class AgentRunEventType(Enum):
+    TEXT = "text"
+    PROGRESS = "progress"
+
+
+class AgentRunEvent(Event):
+    name: str
+    msg: str
+    event_type: AgentRunEventType = AgentRunEventType.TEXT
+    data: Optional[dict] = None
+
+    def to_response(self) -> dict:
+        return {
+            "type": "agent",
+            "data": {
+                "agent": self.name,
+                "type": self.event_type.value,
+                "text": self.msg,
+                "data": self.data,
+            },
+        }
+
+
+class ComponentDefinition(BaseModel):
+    type: str
+    code: str
+    filename: str
+
+
+class UIEvent(Event):
+    type: str
+    data: BaseModel
+
+    def to_response(self) -> dict:
+        return {
+            "type": self.type,
+            "data": self.data.model_dump(),
+        }

python/llama-index-server/llama_index/server/prompts.py

@@ -7,9 +7,46 @@ Here is the conversation history
 ---------------------
 Given the conversation history, please give me 3 questions that user might ask next!
 Your answer should be wrapped in three sticks without any index numbers and follows the following format:
-\`\`\`
+```
 <question 1>
 <question 2>
 <question 3>
-\`\`\`
+```
+"""
+
+# Used as a prompt for synthesizer
+# Override this prompt by setting the `CITATION_PROMPT` environment variable
+CITATION_PROMPT = """
+Context information is below.
+------------------
+{context_str}
+------------------
+The context are multiple text chunks, each text chunk has its own citation_id at the beginning.
+Use the citation_id for citation construction.
+
+Answer the following query with citations:
+------------------
+{query_str}
+------------------
+
+## Citation format
+
+[citation:id]
+
+Where:
+- [citation:] is a matching pattern which is required for all citations.
+- `id` is the `citation_id` provided in the context or previous response.
+
+Example:
+```
+    Here is a response that uses context information [citation:90ca859f-4f32-40ca-8cd0-edfad4fb298b] 
+    and other ideas that don't use context information [citation:17b2cc9a-27ae-4b6d-bede-5ca60fc00ff4] .\n
+    The citation block will be displayed automatically with useful information for the user in the UI [citation:1c606612-e75f-490e-8374-44e79f818d19] .
+```
+
+## Requirements:
+1. Always include citations for every fact from the context information in your response. 
+2. Make sure that the citation_id is correct with the context, don't mix up the citation_id with other information.
+
+Now, you answer the query with citations:
 """

python/llama-index-server/llama_index/server/services/custom_ui.py

@@ -2,7 +2,7 @@ import logging
 import os
 from typing import List, Optional
 
-from llama_index.server.api.models import ComponentDefinition
+from llama_index.server.models.ui import ComponentDefinition
 
 
 class CustomUI:

python/llama-index-server/llama_index/server/services/llamacloud/file.py

@@ -8,11 +8,13 @@ from typing import Any, Dict, List, Optional, Set, Tuple, Union
 import requests
 from fastapi import BackgroundTasks
 from llama_cloud import ManagedIngestionStatus, PipelineFileCreateCustomMetadataValue
-from llama_index.core.schema import NodeWithScore
-from llama_index.server.api.models import SourceNodes
-from llama_index.server.services.llamacloud.index import get_client
 from pydantic import BaseModel
 
+from llama_index.core.schema import NodeWithScore
+from llama_index.server.models.source_nodes import SourceNodes
+from llama_index.server.services.llamacloud.index import get_client
+from llama_index.server.utils import llamacloud
+
 logger = logging.getLogger("uvicorn")
 
 
@@ -33,7 +35,6 @@ class LlamaCloudFile(BaseModel):
 
 class LlamaCloudFileService:
     LOCAL_STORE_PATH = "output/llamacloud"
-    DOWNLOAD_FILE_NAME_TPL = "{pipeline_id}${filename}"
 
     @classmethod
     def get_all_projects_with_pipelines(cls) -> List[Dict[str, Any]]:
@@ -155,13 +156,12 @@ class LlamaCloudFileService:
         # Remove duplicates and return
         return set(llama_cloud_files)
 
-    @classmethod
-    def _get_file_name(cls, name: str, pipeline_id: str) -> str:
-        return cls.DOWNLOAD_FILE_NAME_TPL.format(pipeline_id=pipeline_id, filename=name)
-
     @classmethod
     def _get_file_path(cls, name: str, pipeline_id: str) -> str:
-        return os.path.join(cls.LOCAL_STORE_PATH, cls._get_file_name(name, pipeline_id))
+        file_name = llamacloud.get_local_file_name(
+            llamacloud_file_name=name, pipeline_id=pipeline_id
+        )
+        return os.path.join(cls.LOCAL_STORE_PATH, file_name)
 
     @classmethod
     def _download_file(cls, url: str, local_file_path: str) -> None:

python/llama-index-server/llama_index/server/services/llamacloud/index.py

@@ -3,14 +3,15 @@ import os
 from typing import TYPE_CHECKING, Any, Optional
 
 from llama_cloud import PipelineType
+from pydantic import BaseModel, Field, field_validator
+
 from llama_index.core.callbacks import CallbackManager
 from llama_index.core.ingestion.api_utils import (
     get_client as llama_cloud_get_client,
 )
 from llama_index.core.settings import Settings
 from llama_index.indices.managed.llama_cloud import LlamaCloudIndex
-from llama_index.server.api.models import ChatRequest
-from pydantic import BaseModel, Field, field_validator
+from llama_index.server.models.chat import ChatRequest
 
 if TYPE_CHECKING:
     from llama_cloud.client import LlamaCloud

python/llama-index-server/llama_index/server/services/suggest_next_question.py

@@ -5,7 +5,7 @@ from typing import List, Optional, Union
 
 from llama_index.core.prompts import PromptTemplate
 from llama_index.core.settings import Settings
-from llama_index.server.api.models import ChatAPIMessage
+from llama_index.server.models.chat import ChatAPIMessage
 from llama_index.server.prompts import SUGGEST_NEXT_QUESTION_PROMPT
 
 logger = logging.getLogger("uvicorn")

python/llama-index-server/llama_index/server/services/workflow.py

@@ -0,0 +1,106 @@
+import json
+import logging
+from pathlib import Path
+from typing import Type
+
+from llama_index.core.workflow import (
+    Context,
+    JsonSerializer,
+    Workflow,
+)
+from llama_index.server.models.hitl import HumanResponseEvent
+from llama_index.server.utils.class_meta_serialization import (
+    type_from_identifier,
+    type_identifier,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class HITLWorkflowService:
+    """
+    A service for helping pause and resume a HITL workflow.
+    """
+
+    # A key in context that stores the HITL event type
+    HITL_CONTEXT_KEY = "human_response_type"
+
+    @staticmethod
+    def get_storage_path(id: str) -> Path:
+        storage_dir = Path("output") / "checkpoints"
+        if not storage_dir.exists():
+            storage_dir.mkdir(parents=True, exist_ok=True)
+        return storage_dir / f"{id}.json"
+
+    @classmethod
+    async def save_context(
+        cls,
+        id: str,
+        ctx: Context,
+        resume_event_type: Type[HumanResponseEvent],
+    ) -> None:
+        """
+        Save the current checkpoint to a file and return the id
+
+        Args:
+            id: The id to save the context to.
+            ctx: The context to save.
+            resume_event_type [Optional]: Save workflow context with a resume event.
+        """
+        await ctx.set(
+            key=cls.HITL_CONTEXT_KEY,
+            value=type_identifier(resume_event_type),
+        )
+
+        ctx_data = ctx.to_dict(serializer=JsonSerializer())
+        with open(cls.get_storage_path(id), "w") as f:
+            json.dump(ctx_data, f)
+
+    @classmethod
+    async def load_context(
+        cls,
+        id: str,
+        workflow: Workflow,
+        data: dict,
+    ) -> Context:
+        file_path = cls.get_storage_path(id)
+        if not file_path.exists():
+            raise FileNotFoundError(f"No checkpoint found for id: {id}")
+        try:
+            with open(file_path, "r") as f:
+                ctx_data = json.load(f)
+        except json.JSONDecodeError as e:
+            raise ValueError(f"Invalid checkpoint data for id {id}: {e}")
+        ctx = Context.from_dict(
+            workflow=workflow,
+            data=ctx_data,
+            serializer=JsonSerializer(),
+        )
+        resume_event = await cls._construct_resume_event(ctx, data)
+        ctx.send_event(resume_event)
+        return ctx
+
+    @classmethod
+    async def _construct_resume_event(
+        cls, context: Context, data: dict
+    ) -> HumanResponseEvent:
+        """
+        Get the HITL event from the context.
+        """
+        event_type_str = await context.get(cls.HITL_CONTEXT_KEY)
+        if not event_type_str:
+            raise ValueError(
+                "Cannot resume the workflow because there is no resume event type in the context"
+            )
+        resume_event_type = type_from_identifier(event_type_str)
+        if not issubclass(resume_event_type, HumanResponseEvent):
+            raise ValueError(
+                f"Cannot resume the workflow because the resume event type {resume_event_type} is not a HumanResponseEvent"
+            )
+        try:
+            return resume_event_type(**data)
+        except Exception as e:
+            raise ValueError(
+                f"Error constructing resume event: {e}. "
+                f"Make sure the provided data is valid for the event type {resume_event_type}"
+            )

python/llama-index-server/llama_index/server/tools/index/__init__.py

@@ -1,3 +1,4 @@
+from .citation import CitationSynthesizer, NodeCitationProcessor
 from .query import get_query_engine_tool
 
-__all__ = ["get_query_engine_tool"]
+__all__ = ["get_query_engine_tool", "NodeCitationProcessor", "CitationSynthesizer"]

python/llama-index-server/llama_index/server/tools/index/citation.py

@@ -0,0 +1,72 @@
+import logging
+from typing import Any, List, Optional
+
+from llama_index.core import QueryBundle
+from llama_index.core.postprocessor.types import BaseNodePostprocessor
+from llama_index.core.prompts import PromptTemplate
+from llama_index.core.query_engine.retriever_query_engine import RetrieverQueryEngine
+from llama_index.core.response_synthesizers import Accumulate
+from llama_index.core.schema import NodeWithScore
+from llama_index.core.tools.query_engine import QueryEngineTool
+from llama_index.server.prompts import CITATION_PROMPT
+
+logger = logging.getLogger(__name__)
+
+
+class NodeCitationProcessor(BaseNodePostprocessor):
+    """
+    Add a new field `citation_id` to the metadata of the node by copying the id from the node.
+    Useful for citation construction.
+    """
+
+    def _postprocess_nodes(
+        self,
+        nodes: List[NodeWithScore],
+        query_bundle: Optional[QueryBundle] = None,
+    ) -> List[NodeWithScore]:
+        for node_score in nodes:
+            node_score.node.metadata["citation_id"] = node_score.node.node_id
+        return nodes
+
+
+class CitationSynthesizer(Accumulate):
+    """
+    Overload the Accumulate synthesizer to:
+    1. Update prepare node metadata for citation id
+    2. Update text_qa_template to include citations
+    """
+
+    def __init__(self, **kwargs: Any) -> None:
+        text_qa_template = kwargs.pop("text_qa_template", None)
+        if text_qa_template is None:
+            text_qa_template = PromptTemplate(template=CITATION_PROMPT)
+        super().__init__(text_qa_template=text_qa_template, **kwargs)
+
+
+# Add this prompt to your agent system prompt
+CITATION_SYSTEM_PROMPT = (
+    "\nAnswer the user question using the response from the query tool. "
+    "It's important to respect the citation information in the response. "
+    "Don't mix up the citation_id, keep them at the correct fact."
+)
+
+
+def enable_citation(query_engine_tool: QueryEngineTool) -> QueryEngineTool:
+    """
+    Enable citation for a query engine tool by using CitationSynthesizer and NodePostprocessor.
+    Note: This function will override the response synthesizer of your query engine.
+    """
+    query_engine = query_engine_tool.query_engine
+    if not isinstance(query_engine, RetrieverQueryEngine):
+        raise ValueError(
+            "Citation feature requires a RetrieverQueryEngine. Your tool's query engine is a "
+            f"{type(query_engine)}."
+        )
+    # Update the response synthesizer and node postprocessors
+    query_engine._response_synthesizer = CitationSynthesizer()
+    query_engine._node_postprocessors += [NodeCitationProcessor()]
+    query_engine_tool._query_engine = query_engine
+
+    # Update tool metadata
+    query_engine_tool.metadata.description += "\nThe output will include citations with the format [citation:id] for each chunk of information in the knowledge base."
+    return query_engine_tool

python/llama-index-server/llama_index/server/tools/index/query.py

@@ -1,9 +1,12 @@
+import logging
 import os
 from typing import Any, Optional
 
 from llama_index.core.base.base_query_engine import BaseQueryEngine
-from llama_index.core.tools.query_engine import QueryEngineTool
 from llama_index.core.indices.base import BaseIndex
+from llama_index.core.tools.query_engine import QueryEngineTool
+
+logger = logging.getLogger(__name__)
 
 
 def create_query_engine(index: BaseIndex, **kwargs: Any) -> BaseQueryEngine:
@@ -38,12 +41,11 @@ def get_query_engine_tool(
     if name is None:
         name = "query_index"
     if description is None:
-        description = (
-            "Use this tool to retrieve information about the text corpus from an index."
-        )
+        description = "Use this tool to retrieve information from a knowledge base. Provide a specific query and can call the tool multiple times if necessary."
     query_engine = create_query_engine(index, **kwargs)
-    return QueryEngineTool.from_defaults(
+    tool = QueryEngineTool.from_defaults(
         query_engine=query_engine,
         name=name,
         description=description,
     )
+    return tool

python/llama-index-server/llama_index/server/utils/agent_tool.py

@@ -14,7 +14,7 @@ from llama_index.core.tools import (
     ToolSelection,
 )
 from llama_index.core.workflow import Context
-from llama_index.server.api.models import AgentRunEvent, AgentRunEventType
+from llama_index.server.models.ui import AgentRunEvent, AgentRunEventType
 from llama_index.core.agent.workflow.workflow_events import ToolCall, ToolCallResult
 
 logger = logging.getLogger("uvicorn")

python/llama-index-server/llama_index/server/utils/chat_file.py

@@ -0,0 +1,36 @@
+import os
+from typing import Any, Dict, Optional
+
+from llama_index.server.settings import server_settings
+from llama_index.server.utils import llamacloud
+
+
+def get_file_url_from_metadata(
+    metadata: Dict[str, Any],
+    data_dir: Optional[str] = None,
+) -> Optional[str]:
+    """
+    Get the URL of a file from the source node metadata.
+    """
+    url_prefix = server_settings.file_server_url_prefix
+    if data_dir is None:
+        data_dir = "data"
+    file_name = metadata.get("file_name")
+
+    if file_name and url_prefix:
+        if llamacloud.is_llamacloud_file(metadata):
+            file_name = llamacloud.get_local_file_name(metadata)
+            return f"{url_prefix}/output/llamacloud/{file_name}"
+        is_private = metadata.get("private", "false") == "true"
+        if is_private:
+            # file is a private upload
+            return f"{url_prefix}/output/uploaded/{file_name}"
+        # file is from calling the 'generate' script
+        # Get the relative path of file_path to data_dir
+        file_path = metadata.get("file_path")
+        data_dir = os.path.abspath(data_dir)
+        if file_path and data_dir:
+            relative_path = os.path.relpath(file_path, data_dir)
+            return f"{url_prefix}/data/{relative_path}"
+    # fallback to URL in metadata (e.g. for websites)
+    return metadata.get("URL")