Compare commits

...

23 Commits

Author SHA1 Message Date
github-actions[bot] 05453d55bf Release 0.5.6 (#569)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-16 20:40:15 +07:00
Huu Le d363ced4d8 bump llamaindex server package versions to 0.1.13 (python) and 0.1.3 (ts) (#568) 2025-04-16 20:38:58 +07:00
github-actions[bot] 293c6f97c1 chore(release): bump version to 0.1.13 (#561)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-16 16:29:41 +07:00
Huu Le 44b4d89ac1 Update document link and fix import (#565) 2025-04-16 16:23:17 +07:00
github-actions[bot] 60f10c5b5d Release 0.5.5 (#564)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-15 20:55:53 +07:00
Huu Le ee85320701 fix: missing default export (#563) 2025-04-15 20:54:23 +07:00
github-actions[bot] b12dc6f1e8 Release 0.5.4 (#562)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-15 18:28:11 +07:00
Huu Le 7c3b279417 support code generation of event components using an LLM (Python) (#557)
---------
Co-authored-by: Marcus Schiesser <mail@marcusschiesser.de>
2025-04-15 18:23:06 +07:00
github-actions[bot] 1514a555d5 chore(release): bump version to 0.1.12 (#559)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-15 17:32:13 +07:00
Huu Le cddb4f6bcc chore: bump chat UI version to 0.1.2 and rename generate_ui_for_workflow (#560)
* chore: bump chat UI version to 0.1.2 and rename generate_ui_for_workflow

* feat: add exports for event component generation in gen_ui module

* update document

* refine prompt
2025-04-15 17:27:22 +07:00
github-actions[bot] c82e4f5791 chore(release): bump version to 0.1.11 (#555)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-15 13:11:15 +07:00
Huu Le 1f7e0e3c69 add GenUIWorkflow for generating UI components from workflow events (#549)
* feat: add GenUIWorkflow for generating UI components from workflow events

* feat: enhance GenUIWorkflow to support event handling and UI generation

* add cache, split code

* use gemini model

* refactor: update GenUIWorkflow to use Anthropic model and add pre-run checks for API key and package installation

* feat: introduce PlanningEvent and enhance GenUIWorkflow for improved UI planning and aggregation function generation

* feat: add gen ui to llamaindexserver

* refactor: remove unused gen_ui.py file

* simplify

* update for tailwindcss

* simplify code and add document

* refine text

* feat: add UIEvent model and update exports in server module

* use default UIEvent

* fix wrong model, update template

* add missing doc

* fix linting

* revert change on template

* fix mypy

* disable e2e for the change from llama-index-server

* remove unused script entry from pyproject.toml and refine UI notice text in GenUIWorkflow

* update workflow, bump chat ui

* Refine GenUIWorkflow documentation and improve code structure notes; add llm parameter to generate_ui_for_workflow function.
2025-04-15 13:06:55 +07:00
github-actions[bot] 7997cdeb70 Release 0.5.3 (#556)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-10 19:08:02 +07:00
Huu Le 76ec3605e5 update templates to use new chat UI config (#553) 2025-04-10 19:03:06 +07:00
github-actions[bot] 5cfdec7d75 chore(release): bump version to 0.1.10 (#550)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-10 17:47:23 +07:00
Huu Le 3d1b15d515 fix encoding windows (#554) 2025-04-10 17:37:49 +07:00
Huu Le 392393af9e feat: Add config app title for python, enhance config parameter. (#540)
* Enhance LlamaIndexServer UI configuration

* bump version, add use llamacloud to chat ui config

* add changeset

* refactor: streamline UI configuration and component directory handling

* relock and fix test

* remove change set

* update docs

* fix wrong key name

* fix test

* bump chat ui

* improve docs
2025-04-10 16:45:20 +07:00
Marcus Schiesser 920beda8ad chore: use own DeepResearchEvent (#552) 2025-04-09 20:44:38 +07:00
github-actions[bot] e6f8add778 Release 0.5.2 (#551)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-09 19:40:36 +07:00
Huu Le c9f8f8d5f2 feat: Use custom component for deep research use case (#548) 2025-04-09 19:31:09 +07:00
github-actions[bot] 24eb7736ee chore(release): bump version to 0.1.9 (#545)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-09 19:01:03 +07:00
Huu Le 5fb27220f7 feat: Add componentDir for llama_index_sever (#547)
* init code for custom components

* change router name

* use jsx

* add custom components code

* revert change on create-llama

* fix mypy

* adding document for custom component

* Refactor component directory handling in LlamaIndexServer

* add file name in components response

* Enhance documentation

* fix mypy

* use tmp in test

* docs: word smithing

* Refactor component loading logic in CustomUI to prioritize TSX over JSX files and improve duplicate handling.

* bump chat ui

---------

Co-authored-by: Marcus Schiesser <mail@marcusschiesser.de>
2025-04-09 18:51:39 +07:00
github-actions[bot] 5caa3813f8 chore(release): bump version to 0.1.8 (#534)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-03 21:33:54 +07:00
31 changed files with 2485 additions and 910 deletions
+4
View File
@@ -2,8 +2,12 @@ name: E2E Tests
on:
push:
branches: [main]
paths-ignore:
- "llama-index-server/**"
pull_request:
branches: [main]
paths-ignore:
- "llama-index-server/**"
env:
POETRY_VERSION: "1.6.1"
+30
View File
@@ -1,5 +1,35 @@
# create-llama
## 0.5.6
### Patch Changes
- d363ced: Bump llamaindex server packages
## 0.5.5
### Patch Changes
- ee85320: The default custom deep research component does not work.
## 0.5.4
### Patch Changes
- 7c3b279: Support code generation of event components using an LLM (Python)
## 0.5.3
### Patch Changes
- 76ec360: Update templates to use new chat ui config
## 0.5.2
### Patch Changes
- c9f8f8d: Use custom component for deep research use case
## 0.5.1
### Patch Changes
+6
View File
@@ -536,6 +536,12 @@ const installLlamaIndexServerTemplate = async ({
cwd: path.join(templatesDir, "components", "workflows", "python", useCase),
});
// Copy custom UI component code
await copy(`*`, path.join(root, "components"), {
parents: true,
cwd: path.join(templatesDir, "components", "ui", "workflows", useCase),
});
if (useLlamaParse) {
await copy("index.py", path.join(root, "app"), {
parents: true,
+6
View File
@@ -42,6 +42,12 @@ const installLlamaIndexServerTemplate = async ({
),
});
// copy workflow UI components to output/components folder
await copy("*", path.join(root, "components"), {
parents: true,
cwd: path.join(templatesDir, "components", "ui", "workflows", useCase),
});
if (vectorDb === "llamacloud") {
await copy("generate.ts", path.join(root, "src"), {
parents: true,
+17 -6
View File
@@ -6,7 +6,7 @@ LlamaIndexServer is a FastAPI-based application that allows you to quickly launc
- Serving a workflow as a chatbot
- Built on FastAPI for high performance and easy API development
- Optional built-in chat UI
- Optional built-in chat UI with extendable UI components
- Prebuilt development code
## Installation
@@ -43,8 +43,10 @@ def create_workflow() -> Workflow:
app = LlamaIndexServer(
workflow_factory=create_workflow, # Supports Workflow or AgentWorkflow
env="dev", # Enable development mode
include_ui=True, # Include chat UI
starter_questions=["What can you do?", "How do I use this?"],
ui_config={ # Configure the chat UI, optional
"app_title": "Weather Bot",
"starter_questions": ["What is the weather in LA?", "Will it rain in SF?"],
},
verbose=True
)
```
@@ -74,12 +76,16 @@ The LlamaIndexServer accepts the following configuration parameters:
- `logger`: Optional logger instance (defaults to uvicorn logger)
- `use_default_routers`: Whether to include default routers (chat, static file serving)
- `env`: Environment setting ('dev' enables CORS and UI by default)
- `include_ui`: Whether to include the chat UI
- `starter_questions`: List of starter questions for the chat UI
- `ui_config`: UI configuration as a dictionary or UIConfig object with options:
- `enabled`: Whether to enable the chat UI (default: True)
- `app_title`: The title of the chat application (default: "LlamaIndex Server")
- `starter_questions`: List of starter questions for the chat UI (default: None)
- `ui_path`: Path for downloaded UI static files (default: ".ui")
- `component_dir`: The directory for custom UI components rendering events emitted by the workflow. The default is None, which does not render custom UI components.
- `llamacloud_index_selector`: Whether to show the LlamaCloud index selector in the chat UI (default: False). Requires `LLAMA_CLOUD_API_KEY` to be set.
- `verbose`: Enable verbose logging
- `api_prefix`: API route prefix (default: "/api")
- `server_url`: The deployment URL of the server (default is None)
- `ui_path`: Path for downloaded UI static files (default: ".ui")
## Default Routers and Features
@@ -101,6 +107,11 @@ When enabled, the server provides a chat interface at the root path (`/`) with:
- Real-time chat interface
- API endpoint integration
### Custom UI Components
You can add custom UI components for your workflow by providing `component_dir` config and adding custom .jsx or .tsx files to the directory.
See [Custom UI Components](https://github.com/run-llama/create-llama/blob/main/llama-index-server/docs/custom_ui_component.md) for more details.
## Development Mode
In development mode (`env="dev"`), the server:
@@ -0,0 +1,103 @@
# Custom UI Components
The LlamaIndex server provides support for rendering workflow events using custom UI components, allowing you to extend and customize the chat interface.
## Overview
Custom UI components are a powerful feature that enables you to:
- Add custom interface elements to the chat UI using React JSX or TSX files
- Extend the default chat interface functionality
- Create specialized visualizations or interactions
## Configuration
### Workflow events
To display custom UI components, your workflow needs to emit `UIEvent` events with data that conforms to the data model of your custom UI component.
```python
from llama_index.server import UIEvent
from pydantic import BaseModel, Field
from typing import Literal, Any
# Define a Pydantic model for your event data
class DeepResearchEventData(BaseModel):
id: str = Field(description="The unique identifier for the event")
type: Literal["retrieval", "analysis"] = Field(description="DeepResearch has two main stages: retrieval and analysis")
status: Literal["pending", "completed", "failed"] = Field(description="The current status of the event")
content: str = Field(description="The textual content of the event")
# In your workflow, emit the data model with UIEvent
ctx.write_event_to_stream(
UIEvent(
type="deep_research_event",
data=DeepResearchEventData(
id="123",
type="retrieval",
status="pending",
content="Retrieving data...",
),
)
)
```
### Server Setup
1. Initialize the LlamaIndex server with a component directory:
```python
server = LlamaIndexServer(
workflow_factory=your_workflow,
ui_config={
"component_dir": "path/to/components",
},
include_ui=True
)
```
2. Add the custom component code to the directory following the naming pattern:
- File Extension: `.jsx` and `.tsx` for React components
- File Name: Should match the event type from your workflow (e.g., `deep_research_event.jsx` for handling `deep_research_event` type that you defined in your workflow). If there are TSX and JSX files with the same name, the TSX file will be used.
- Component Name: Export a default React component named `Component` that receives props from the event data
Example component structure:
```jsx
function Component({ events }) {
// Your component logic here
return (
// Your UI code here
);
}
```
### Generate UI Component
We provide a `generate_event_component` function that uses LLMs to automatically generate UI components for your workflow events.
```python
from llama_index.server.gen_ui import generate_event_component
from llama_index.llms.openai import OpenAI
# Generate a component using the event class you defined in your workflow
from your_workflow import DeepResearchEvent
ui_code = await generate_event_component(
event_cls=DeepResearchEvent,
llm=OpenAI(model="gpt-4.1"), # Default LLM is Claude 3.7 Sonnet if not provided
)
# Alternatively, generate from your workflow file
ui_code = await generate_event_component(
workflow_file="your_workflow.py",
)
print(ui_code)
# Save the generated code to a file for use in your project
with open("deep_research_event.jsx", "w") as f:
f.write(ui_code)
```
> **Tip:** For optimal results, add descriptive documentation to each field in your event data class. This helps the LLM better understand your data structure and generate more appropriate UI components. We also recommend using GPT 4.1, Claude 3.7 Sonnet and Gemini 2.5 Pro for better results.
@@ -1,3 +1,4 @@
from .server import LlamaIndexServer
from .api.models import UIEvent
from .server import LlamaIndexServer, UIConfig
__all__ = ["LlamaIndexServer"]
__all__ = ["LlamaIndexServer", "UIConfig", "UIEvent"]
@@ -134,3 +134,20 @@ class SourceNodes(BaseModel):
cls, source_nodes: List[NodeWithScore]
) -> List["SourceNodes"]:
return [cls.from_source_node(node) for node in source_nodes]
class ComponentDefinition(BaseModel):
type: str
code: str
filename: str
class UIEvent(Event):
type: str
data: BaseModel
def to_response(self) -> dict:
return {
"type": self.type,
"data": self.data.model_dump(),
}
@@ -0,0 +1,4 @@
from llama_index.server.api.routers.chat import chat_router
from llama_index.server.api.routers.ui import custom_components_router
__all__ = ["chat_router", "custom_components_router"]
@@ -6,6 +6,7 @@ from typing import AsyncGenerator, Callable, Union
from fastapi import APIRouter, BackgroundTasks, HTTPException
from fastapi.responses import StreamingResponse
from llama_index.core.agent.workflow.workflow_events import AgentStream
from llama_index.core.workflow import StopEvent, Workflow
from llama_index.server.api.callbacks import (
@@ -99,7 +100,10 @@ async def _stream_content(
event: Union[AgentStream, StopEvent],
) -> AsyncGenerator[str, None]:
if isinstance(event, AgentStream):
yield event.delta
# Normally, if the stream is a tool call, the delta is always empty
# so it's not a text stream.
if len(event.tool_calls) == 0:
yield event.delta
elif isinstance(event, StopEvent):
if isinstance(event.result, str):
yield event.result
@@ -0,0 +1,20 @@
import logging
from typing import List
from fastapi import APIRouter
from llama_index.server.api.models import ComponentDefinition
from llama_index.server.services.custom_ui import CustomUI
def custom_components_router(
component_dir: str,
logger: logging.Logger,
) -> APIRouter:
router = APIRouter(prefix="/components")
@router.get("")
async def components() -> List[ComponentDefinition]:
custom_ui = CustomUI(component_dir=component_dir, logger=logger)
return custom_ui.get_components()
return router
@@ -5,7 +5,7 @@ from typing import Optional
import requests
CHAT_UI_VERSION = "0.0.6"
CHAT_UI_VERSION = "0.1.2"
def download_chat_ui(
@@ -0,0 +1,4 @@
from .main import generate_event_component
from .parse_workflow_code import get_workflow_event_schemas
__all__ = ["generate_event_component", "get_workflow_event_schemas"]
@@ -0,0 +1,442 @@
import re
from typing import Any, Dict, List, Optional, Type
from pydantic import BaseModel
from rich.console import Console
from rich.live import Live
from rich.panel import Panel
from llama_index.core.llms import LLM
from llama_index.core.prompts import PromptTemplate
from llama_index.core.workflow import (
Context,
Event,
StartEvent,
StopEvent,
Workflow,
step,
)
from llama_index.server.gen_ui.parse_workflow_code import get_workflow_event_schemas
class PlanningEvent(Event):
"""
Event for planning the UI.
"""
events: List[Dict[str, Any]]
class WriteAggregationEvent(Event):
"""
Event for aggregating events.
"""
events: List[Dict[str, Any]]
ui_description: str
class WriteUIComponentEvent(Event):
"""
Event for writing UI component.
"""
events: List[Dict[str, Any]]
aggregation_function: Optional[str]
ui_description: str
class RefineGeneratedCodeEvent(Event):
"""
Refine the generated code.
"""
generated_code: str
aggregation_function_context: Optional[str]
events: List[Dict[str, Any]]
class ExtractEventSchemaEvent(Event):
"""
Extract the event schema from the event.
"""
events: List[Any]
class AggregatePrediction(BaseModel):
"""
Prediction for aggregating events or not.
If need_aggregation is True, the aggregation_function will be provided.
"""
need_aggregation: bool
aggregation_function: Optional[str]
class GenUIWorkflow(Workflow):
"""
Generate UI component for event from workflow.
"""
code_structure: str = """
```jsx
// Note: Only React, shadcn/ui, lucide-react, LlamaIndex's markdown-ui and tailwind css (cn) are allowed.
// export the component
export default function Component({ events }) {
// logic for aggregating events (if needed)
const aggregateEvents = () => {
// code for aggregating events here
}
// State handling
// e.g: const [state, setState] = useState({});
return (
// UI code here
)
}
```
"""
supported_deps = """
- React: import { useState } from "react";
- shadcn/ui: import { ComponentName } from "@/components/ui/<component_path>";
Supported shadcn components:
accordion, alert, alert-dialog, aspect-ratio, avatar, badge,
breadcrumb, button, calendar, card, carousel, chart, checkbox, collapsible, command,
context-menu, dialog, drawer, dropdown-menu, form, hover-card, input, input-otp, label,
menubar, navigation-menu, pagination, popover, progress, radio-group, resizable,
scroll-area, select, separator, sheet, sidebar, skeleton, slider, sonner, switch, table,
tabs, textarea, toggle, toggle-group, tooltip
- lucide-react: import { IconName } from "lucide-react";
- tailwind css: import { cn } from "@/lib/utils"; // Note: clsx is not supported
- LlamaIndex's markdown-ui: import { Markdown } from "@llamaindex/chat-ui/widgets";
"""
def __init__(self, llm: LLM, **kwargs: Any):
super().__init__(**kwargs)
self.llm = llm
self.console = Console()
self._live: Optional[Live] = None
self._completed_steps: List[str] = []
self._current_step: Optional[str] = None
def update_status(self, message: str, completed: bool = False) -> None:
"""Show completed and current steps in a panel."""
if completed:
if self._current_step:
self._completed_steps.append(self._current_step)
self._current_step = None
else:
self._current_step = message
if self._live is None:
self._live = Live("", console=self.console, refresh_per_second=4)
self._live.start()
# Build status display
status_lines = []
for completed_step in self._completed_steps:
status_lines.append(f"[green]✓[/green] {completed_step}")
if self._current_step:
status_lines.append(f"[yellow]⋯[/yellow] {self._current_step}")
self._live.update(Panel("\n".join(status_lines)))
@step
async def start(self, ctx: Context, ev: StartEvent) -> PlanningEvent:
events = ev.events
if not events:
raise ValueError(
"events is required, provide list of filtered events to generate UI components for"
)
await ctx.set("events", events)
self.update_status("Planning the UI")
return PlanningEvent(events=events)
@step
async def planning(self, ctx: Context, ev: PlanningEvent) -> WriteAggregationEvent:
prompt_template = """
# Your role
You are a designer who is designing a UI for given events that are emitted from a backend workflow.
Here are the events that you need to work on: {events}
# Task
Your task is to analyze the event schema and data and provide a description that how the UI would look like.
The UI should be beautiful, no monotonous, and visually pleasing.
Focus on the elements and the layout, don't ask too much on the styles (transition, dark mode, responsive, etc...).
e.g: Assume that the backend produce list of events with animal name, action, and status.
```
A card-based layout displaying animal actions:
- Each card shows an animal's image at the top
- Below the image: animal name as the card title
- Action details in the card body with an icon (eating 🍖, sleeping 😴, playing 🎾)
- Status badge in the corner showing if action is ongoing/completed
- Expandable section for additional details
- Soft color scheme based on action type
```
Don't be verbose, just return the description for the UI based on the event schema and data.
"""
response = await self.llm.acomplete(
PromptTemplate(prompt_template).format(events=ev.events),
formatted=True,
)
await ctx.set("ui_description", response.text)
self.update_status("Planning the UI", completed=True)
# Update the planning description to the console
self.console.print(
Panel(
response.text,
title="UI Description",
border_style="cyan",
)
)
self.update_status("Generating aggregation function")
return WriteAggregationEvent(
events=ev.events,
ui_description=response.text,
)
@step
async def generate_event_aggregations(
self, ctx: Context, ev: WriteAggregationEvent
) -> WriteUIComponentEvent:
prompt_template = """
# Your role
You are a frontend developer who is developing a React component for given events that are emitted from a backend workflow.
Here are the events that you need to work on: {events}
Here is the description of the UI:
```
{ui_description}
```
# Task
Based on the description of the UI and the list of events, write the aggregation function that will be used to aggregate the events.
Take into account that the list of events grows with time. At the beginning, there is only one event in the list, and events are incrementally added.
To render the events in a visually pleasing way, try to aggregate them by their attributes and render the aggregates instead of just rendering a list of all events.
Don't add computation to the aggregation function, just group the events by their attributes.
Make sure that the aggregation should reflect the description of the UI and the grouped events are not duplicated, make it as simple as possible to avoid unnecessary issues.
# Answer with the following format:
```jsx
const aggregateEvents = () => {
// code for aggregating events here if needed otherwise let the jsx code block empty
}
```
"""
response = await self.llm.acomplete(
PromptTemplate(prompt_template).format(
events=ev.events,
ui_description=ev.ui_description,
),
formatted=True,
)
await ctx.set("aggregation_context", response.text)
self.update_status("Generating aggregation function", completed=True)
self.update_status("Generating UI components")
return WriteUIComponentEvent(
events=ev.events,
aggregation_function=response.text,
ui_description=ev.ui_description,
)
@step
async def write_ui_component(
self, ctx: Context, ev: WriteUIComponentEvent
) -> RefineGeneratedCodeEvent:
prompt_template = """
# Your role
You are a frontend developer who is developing a React component using shadcn/ui, lucide-react, LlamaIndex's chat-ui, and tailwind css (cn) for the UI.
You are given a list of events and other context.
Your task is to write a beautiful UI for the events that will be included in a chat UI.
# Context:
Here are the events that you need to work on: {events}
{aggregation_function_context}
Here is the description of the UI:
```
{ui_description}
```
# Supported dependencies:
{supported_deps}
# Requirements:
- Write beautiful UI components for the events using the supported dependencies
- The component text/label should be specified for each event type.
# Instructions:
## Event and schema notice
- Based on the provided list of events, determine their types and attributes.
- It's normal that the schema is applied to all events, but the events might completely different which some of schema attributes aren't used.
- You should make the component visually distinct for each event type.
e.g: A simple cat schema
```{"type": "cat", "action": ["jump", "run", "meow"], "jump": {"height": 10, "distance": 20}, "run": {"distance": 100}}```
You should display the jump, run and meow actions in different ways. don't try to render "height" for the "run" and "meow" action.
## UI notice
- Use the supported dependencies for the UI.
- Be careful on state handling, make sure the update should be updated in the state and there is no duplicate state.
- For a long content, consider to use markdown along with dropdown to show the full content.
e.g:
```jsx
import { Markdown } from "@llamaindex/chat-ui/widgets";
<Markdown content={content} />
```
- Try to make the component placement not monotonous, consider use row/column/flex/grid layout.
"""
aggregation_function_context = (
f"\nBefore rendering the events, we're using the following aggregation function: {ev.aggregation_function}"
if ev.aggregation_function
else ""
)
prompt = PromptTemplate(prompt_template).format(
events=ev.events,
aggregation_function_context=aggregation_function_context,
code_structure=self.code_structure,
ui_description=ev.ui_description,
supported_deps=self.supported_deps,
)
response = await self.llm.acomplete(prompt, formatted=True)
self.update_status("Generating UI components", completed=True)
self.update_status("Refining generated code")
return RefineGeneratedCodeEvent(
generated_code=response.text,
events=ev.events,
aggregation_function_context=aggregation_function_context,
)
@step
async def refine_code(
self, ctx: Context, ev: RefineGeneratedCodeEvent
) -> StopEvent:
prompt_template = """
# Your role
You are a frontend developer who is developing a React component for given events that are emitted from a backend workflow.
Your task is to assemble the pieces of code into a complete code segment that follows the specified code structure.
# Context:
## Here is the generated code:
{generated_code}
{aggregation_function_context}
## The generated code should follow the following structure:
{code_structure}
# Requirements:
- Only use supported dependencies: {supported_deps}
- Refine the code if needed to ensure there are no potential bugs.
- Be careful on code placement, make sure it doesn't call any undefined code.
- Make sure the import statements are correct.
e.g: import { Button, Card, Accordion } from "@/components/ui" is correct because Button, Card are defined in different shadcn/ui components.
-> correction: import { Button } from "@/components/ui/button";
import { Card } from "@/components/ui/card";
- Don't be verbose, only return the code, wrap it in ```jsx <code>```
"""
prompt = PromptTemplate(prompt_template).format(
generated_code=ev.generated_code,
code_structure=self.code_structure,
aggregation_function_context=ev.aggregation_function_context,
supported_deps=self.supported_deps,
)
response = await self.llm.acomplete(prompt, formatted=True)
# Extract code from response, handling case where code block is missing
code_match = re.search(r"```jsx(.*)```", response.text, re.DOTALL)
if code_match is None:
# If no code block found, use full response
code = response.text
else:
code = code_match.group(1).strip()
self.update_status("Refining generated code", completed=True)
if self._live is not None:
self._live.stop()
self._live = None
return StopEvent(
result=code,
)
async def generate_event_component(
workflow_file: Optional[str] = None,
event_cls: Optional[Type[BaseModel]] = None,
llm: Optional[LLM] = None,
) -> str:
"""
Generate UI component for events from workflow.
Either workflow_file or event_cls must be provided.
Args:
workflow_file: The path to the workflow file that contains the event to generate UI for. e.g: `app/workflow.py`.
event_cls: A Pydantic class to generate UI for. e.g: `DeepResearchEvent`.
llm: The LLM to use for the generation. Default is Anthropic's Claude 3.7 Sonnet.
We recommend using these LLMs:
- Anthropic's Claude 3.7 Sonnet
- OpenAI's GPT-4.1
- Google Gemini 2.5 Pro
Returns:
The generated UI component code.
"""
if workflow_file is None and event_cls is None:
raise ValueError(
"Either workflow_file or event_cls must be provided. Please provide one of them."
)
if workflow_file is not None and event_cls is not None:
raise ValueError(
"Only one of workflow_file or event_cls can be provided. Please provide only one of them."
)
if llm is None:
from llama_index.llms.anthropic import Anthropic
llm = Anthropic(model="claude-3-7-sonnet-latest", max_tokens=8192)
console = Console()
# Get event schemas
if workflow_file is not None:
# Get event schemas from the input file
console.rule("[bold blue]Analyzing Events[/bold blue]")
event_schemas = get_workflow_event_schemas(workflow_file)
if len(event_schemas) == 0:
console.print(
Panel(
"[red]No events found that are used with write_event_to_stream[/red]",
title="❌ Error",
border_style="red",
)
)
raise RuntimeError(
"No events found that are used with write_event_to_stream. Please check the workflow file."
)
elif event_cls is not None:
event_schemas = [
{"type": event_cls.__name__, "schema": event_cls.model_json_schema()}
]
# Generate UI component from event schemas
console.rule("[bold blue]Generate UI Components[/bold blue]")
workflow = GenUIWorkflow(llm=llm, timeout=500.0)
code = await workflow.run(events=event_schemas)
console.print(
Panel(
"[green]UI component has been generated successfully![/green]\n",
title="✨ Complete",
border_style="green",
)
)
return code
@@ -0,0 +1,93 @@
import ast
import importlib
import inspect
import os
import sys
from typing import Any, Dict, List
class EventAnalyzer(ast.NodeVisitor):
"""
Parse the workflow code to find UIEvent instances passed to write_event_to_stream.
"""
def __init__(self) -> None:
self.found_ui_event = False
def visit_Call(self, node: ast.Call) -> None:
# Check for ctx.write_event_to_stream call with UIEvent arg
if (
isinstance(node.func, ast.Attribute)
and isinstance(node.func.value, ast.Name)
and node.func.attr == "write_event_to_stream"
and node.args
and isinstance(node.args[0], ast.Call)
and isinstance(node.args[0].func, ast.Name)
and node.args[0].func.id == "UIEvent"
):
self.found_ui_event = True
self.generic_visit(node)
def get_workflow_event_schemas(file_path: str) -> List[Dict[str, Any]]:
"""
Find UIEvent instances passed to write_event_to_stream and return their data type schema.
"""
# Get absolute path for module importing
abs_file_path = os.path.abspath(file_path)
project_root = os.path.dirname(os.path.dirname(abs_file_path))
# Convert file path to module name
rel_path = os.path.relpath(abs_file_path, project_root)
module_name = rel_path.replace(os.sep, ".").replace(".py", "")
# Temporarily modify sys.path to allow imports
original_path = list(sys.path)
if project_root not in sys.path:
sys.path.insert(0, project_root)
try:
# Import the module
module = importlib.import_module(module_name)
importlib.reload(module)
except ImportError as e:
print(f"Error importing module {module_name}: {e}")
sys.path = original_path
return []
finally:
# Restore original path
if project_root in sys.path and project_root not in original_path:
sys.path.remove(project_root)
# Parse the file to check for UIEvent usage
try:
with open(file_path, "r") as f:
tree = ast.parse(f.read())
except (FileNotFoundError, SyntaxError) as e:
print(f"Error parsing {file_path}: {e}")
return []
# Check if UIEvent is passed to write_event_to_stream
analyzer = EventAnalyzer()
analyzer.visit(tree)
schema_list = []
# Only proceed if UIEvent was found and the module has the class
if analyzer.found_ui_event and hasattr(module, "UIEvent"):
# Look for class names containing "EventData" in the module
for name, obj in inspect.getmembers(module):
if (
inspect.isclass(obj)
and name.endswith("EventData")
and hasattr(obj, "model_json_schema")
):
try:
schema = obj.model_json_schema()
if schema:
schema_list.append(schema)
except Exception:
pass
return schema_list
+82 -36
View File
@@ -1,23 +1,58 @@
import json
import logging
import os
from typing import Any, Callable, Optional
from typing import Any, Callable, Optional, Union
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from llama_index.core.workflow import Workflow
from llama_index.server.api.routers.chat import chat_router
from llama_index.server.api.routers import chat_router, custom_components_router
from llama_index.server.chat_ui import download_chat_ui
from llama_index.server.settings import server_settings
from pydantic import BaseModel, Field
class UIConfig(BaseModel):
enabled: bool = Field(default=True, description="Whether to enable the chat UI")
app_title: str = Field(
default="LlamaIndex Server", description="The title of the chat UI"
)
starter_questions: Optional[list[str]] = Field(
default=None, description="The starter questions for the chat UI"
)
llamacloud_index_selector: bool = Field(
default=False,
description="Whether to show the LlamaCloud index selector in the chat UI (need to set the LLAMA_CLOUD_API_KEY environment variable)",
)
ui_path: str = Field(
default=".ui", description="The path that stores static files for the chat UI"
)
component_dir: Optional[str] = Field(
default=None, description="The directory to custom UI components code"
)
def get_config_content(self) -> str:
return json.dumps(
{
"CHAT_API": f"{server_settings.api_url}/chat",
"STARTER_QUESTIONS": self.starter_questions or [],
"LLAMA_CLOUD_API": f"{server_settings.api_url}/chat/config/llamacloud"
if self.llamacloud_index_selector and os.getenv("LLAMA_CLOUD_API_KEY")
else None,
"APP_TITLE": self.app_title,
"COMPONENTS_API": f"{server_settings.api_url}/components"
if self.component_dir
else None,
},
indent=2,
)
class LlamaIndexServer(FastAPI):
workflow_factory: Callable[..., Workflow]
include_ui: Optional[bool]
starter_questions: Optional[list[str]]
verbose: bool = False
ui_path: str = ".ui"
ui_config: UIConfig
def __init__(
self,
@@ -25,8 +60,7 @@ class LlamaIndexServer(FastAPI):
logger: Optional[logging.Logger] = None,
use_default_routers: Optional[bool] = True,
env: Optional[str] = None,
include_ui: Optional[bool] = None,
starter_questions: Optional[list[str]] = None,
ui_config: Optional[Union[UIConfig, dict]] = None,
server_url: Optional[str] = None,
api_prefix: Optional[str] = None,
verbose: bool = False,
@@ -41,8 +75,7 @@ class LlamaIndexServer(FastAPI):
logger: The logger to use.
use_default_routers: Whether to use the default routers (chat, mount `data` and `output` directories).
env: The environment to run the server in.
include_ui: Whether to show an chat UI in the root path.
starter_questions: A list of starter questions to display in the chat UI.
ui_config: The configuration for the chat UI.
server_url: The URL of the server.
api_prefix: The prefix for the API endpoints.
verbose: Whether to show verbose logs.
@@ -52,9 +85,13 @@ class LlamaIndexServer(FastAPI):
self.workflow_factory = workflow_factory
self.logger = logger or logging.getLogger("uvicorn")
self.verbose = verbose
self.include_ui = include_ui # Store the explicitly passed value first
self.starter_questions = starter_questions
self.use_default_routers = use_default_routers or True
if ui_config is None:
self.ui_config = UIConfig()
elif isinstance(ui_config, dict):
self.ui_config = UIConfig(**ui_config)
else:
self.ui_config = ui_config
# Update the settings
if server_url:
@@ -67,27 +104,15 @@ class LlamaIndexServer(FastAPI):
if str(env).lower() == "dev":
self.allow_cors("*")
if self.include_ui is None:
self.include_ui = True
if self.include_ui is None:
self.include_ui = False
if self.ui_config.enabled is None:
self.ui_config.enabled = True
if self.include_ui:
if self.ui_config.enabled is None:
self.ui_config.enabled = False
if self.ui_config.enabled:
self.mount_ui()
@property
def _ui_config(self) -> dict:
config = {
"CHAT_API": f"{server_settings.api_url}/chat",
"STARTER_QUESTIONS": self.starter_questions,
}
is_llamacloud_configured = os.getenv("LLAMA_CLOUD_API_KEY") is not None
if is_llamacloud_configured:
config["LLAMA_CLOUD_API"] = (
f"{server_settings.api_url}/chat/config/llamacloud"
)
return config
# Default routers
def add_default_routers(self) -> None:
self.add_chat_router()
@@ -106,18 +131,39 @@ class LlamaIndexServer(FastAPI):
prefix=server_settings.api_prefix,
)
def add_components_router(self) -> None:
"""
Add the UI router.
"""
if self.ui_config.component_dir is None:
raise ValueError("component_dir must be specified to add components router")
self.include_router(
custom_components_router(self.ui_config.component_dir, self.logger),
prefix=server_settings.api_prefix,
)
def mount_ui(self) -> None:
"""
Mount the UI.
"""
# Check if the static folder exists
if self.include_ui:
if not os.path.exists(self.ui_path):
if self.ui_config.enabled:
# Component dir
if self.ui_config.component_dir:
if not os.path.exists(self.ui_config.component_dir):
os.makedirs(self.ui_config.component_dir)
self.add_components_router()
# UI static files
if not os.path.exists(self.ui_config.ui_path):
os.makedirs(self.ui_config.ui_path)
self.logger.warning(
f"UI files not found, downloading UI to {self.ui_path}"
f"UI files not found, downloading UI to {self.ui_config.ui_path}"
)
download_chat_ui(logger=self.logger, target_path=self.ui_path)
self._mount_static_files(directory=self.ui_path, path="/", html=True)
download_chat_ui(logger=self.logger, target_path=self.ui_config.ui_path)
self._mount_static_files(
directory=self.ui_config.ui_path, path="/", html=True
)
self._override_ui_config()
def _override_ui_config(self) -> None:
@@ -125,12 +171,12 @@ class LlamaIndexServer(FastAPI):
Override the UI config by writing a complete configuration file.
"""
try:
config_path = os.path.join(self.ui_path, "config.js")
config_path = os.path.join(self.ui_config.ui_path, "config.js")
if not os.path.exists(config_path):
self.logger.error("Config file not found")
return
config_content = (
f"window.LLAMAINDEX = {json.dumps(self._ui_config, indent=2)};"
f"window.LLAMAINDEX = {self.ui_config.get_config_content()};"
)
with open(config_path, "w") as f:
f.write(config_content)
@@ -0,0 +1,81 @@
import logging
import os
from typing import List, Optional
from llama_index.server.api.models import ComponentDefinition
class CustomUI:
def __init__(
self, component_dir: str, logger: Optional[logging.Logger] = None
) -> None:
self.component_dir = component_dir
self.logger = logger or logging.getLogger(__name__)
def get_components(self) -> List[ComponentDefinition]:
"""
List all js files in the component directory and return a list of ComponentDefinition objects.
Ignores files that fail to load and logs the error.
TSX files take precedence over JSX files when duplicate component names are found.
"""
components_dict: dict[str, ComponentDefinition] = {}
if not os.path.exists(self.component_dir):
self.logger.warning(
f"Component directory {self.component_dir} does not exist"
)
return []
try:
for file in os.listdir(self.component_dir):
if not file.endswith((".jsx", ".tsx")):
continue
component_name = file.split(".")[0]
file_path = os.path.join(self.component_dir, file)
file_ext = os.path.splitext(file)[1]
try:
with open(file_path, "r", encoding="utf-8") as f:
code = f.read()
new_component = ComponentDefinition(
type=component_name,
code=code,
filename=file,
)
if component_name in components_dict:
existing_ext = os.path.splitext(
components_dict[component_name].filename
)[1]
# If existing is TSX and new is JSX, skip and warn
if existing_ext == ".tsx" and file_ext == ".jsx":
self.logger.warning(
f"Skipping duplicate JSX component {file} as TSX version already exists"
)
continue
# If both are same extension, warn and skip
if existing_ext == file_ext:
self.logger.warning(
f"Skipping duplicate component {file} with same extension"
)
continue
# If existing is JSX and new is TSX, replace and warn
if existing_ext == ".jsx" and file_ext == ".tsx":
self.logger.warning(
f"Replacing JSX component {components_dict[component_name].filename} with TSX version {file}"
)
components_dict[component_name] = new_component
continue
components_dict[component_name] = new_component
except Exception as e:
self.logger.error(f"Failed to load component {file}: {str(e)}")
continue
except Exception as e:
self.logger.error(f"Error reading component directory: {str(e)}")
return list(components_dict.values())
+778 -750
View File
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -26,7 +26,7 @@ license = "MIT"
name = "llama-index-server"
packages = [{include = "llama_index/"}]
readme = "README.md"
version = "0.1.7"
version = "0.1.13"
[tool.poetry.dependencies]
python = ">=3.9,<4.0"
@@ -34,7 +34,7 @@ fastapi = {extras = ["standard"], version = "^0.115.11"}
cachetools = "^5.5.2"
requests = "^2.32.3"
pydantic-settings = "^2.8.1"
llama-index-core = "0.12.28"
llama-index-core = "^0.12.28"
llama-index-readers-file = "^0.4.6"
llama-index-indices-managed-llama-cloud = "0.6.3"
@@ -62,3 +62,4 @@ types-setuptools = "67.1.0.0"
xhtml2pdf = "^0.2.17"
pytest-cov = "^6.0.0"
llama-cloud = "^0.1.17"
@@ -1,9 +1,12 @@
import json
import os
import shutil
import pytest
from httpx import ASGITransport, AsyncClient
from llama_index.core.agent.workflow import AgentWorkflow
from llama_index.core.llms import MockLLM
from llama_index.server import LlamaIndexServer
from llama_index.server import LlamaIndexServer, UIConfig
def fetch_weather(city: str) -> str:
@@ -36,7 +39,7 @@ def server() -> LlamaIndexServer:
@pytest.mark.asyncio()
async def test_server_has_chat_route(server: LlamaIndexServer) -> None:
"""Test that the server has the chat API route."""
chat_route_exists = any(route.path == "/api/chat" for route in server.routes)
chat_route_exists = any("/api/chat" in str(route) for route in server.routes)
assert chat_route_exists, "Chat API route not found in server routes"
@@ -57,20 +60,22 @@ async def test_ui_is_downloaded(server: LlamaIndexServer) -> None:
"""
Test if the UI is downloaded and mounted correctly.
"""
import os
import shutil
# Clean up any existing static directory first
if os.path.exists(".ui"):
shutil.rmtree(".ui")
# Create a new server with UI enabled
ui_config = UIConfig(
enabled=True,
app_title="Test UI",
starter_questions=["What's the weather like?"],
)
ui_server = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
use_default_routers=True,
env="dev",
include_ui=True,
ui_config=ui_config,
)
# Verify that static directory was created with index.html
@@ -78,6 +83,21 @@ async def test_ui_is_downloaded(server: LlamaIndexServer) -> None:
assert os.path.isdir("./.ui"), "Static path is not a directory"
assert os.path.exists("./.ui/index.html"), "index.html was not downloaded"
# Check if the config.js was created with correct content
config_path = os.path.join(".ui", "config.js")
assert os.path.exists(config_path), "config.js was not created"
with open(config_path, "r") as f:
config_content = f.read()
assert "window.LLAMAINDEX =" in config_content
config_json = json.loads(
config_content.replace("window.LLAMAINDEX = ", "").rstrip(";")
)
assert config_json["CHAT_API"] == "/api/chat"
assert config_json["STARTER_QUESTIONS"] == ["What's the weather like?"]
assert config_json["LLAMA_CLOUD_API"] is None
assert config_json["APP_TITLE"] == "Test UI"
# Check if the UI is mounted and accessible
async with AsyncClient(
transport=ASGITransport(app=ui_server), base_url="http://test"
@@ -104,3 +124,175 @@ async def test_ui_is_accessible(server: LlamaIndexServer) -> None:
response = await ac.get("/")
assert response.status_code == 200
assert "text/html" in response.headers["content-type"]
@pytest.mark.asyncio()
async def test_ui_config_customization() -> None:
"""
Test if UI configuration can be customized.
"""
custom_config = UIConfig(
enabled=True,
app_title="Custom App",
starter_questions=["Question 1", "Question 2"],
ui_path=".custom_ui",
)
server = LlamaIndexServer(
workflow_factory=_agent_workflow, verbose=True, ui_config=custom_config
)
assert server.ui_config.app_title == "Custom App"
assert server.ui_config.starter_questions == ["Question 1", "Question 2"]
assert server.ui_config.ui_path == ".custom_ui"
# Clean up if directory was created
if os.path.exists(".custom_ui"):
shutil.rmtree(".custom_ui")
@pytest.mark.asyncio()
async def test_ui_config_from_dict() -> None:
"""
Test if UI configuration can be initialized from a dictionary.
"""
ui_config_dict = {
"enabled": True,
"app_title": "Dict Config App",
"starter_questions": ["Dict Q1", "Dict Q2"],
"ui_path": ".dict_ui",
}
server = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
ui_config=ui_config_dict,
)
# Verify the config was properly converted to UIConfig object
assert isinstance(server.ui_config, UIConfig)
assert server.ui_config.app_title == "Dict Config App"
assert server.ui_config.starter_questions == ["Dict Q1", "Dict Q2"]
assert server.ui_config.ui_path == ".dict_ui"
# Verify the config.js is created with correct content
server.mount_ui()
config_path = os.path.join(".dict_ui", "config.js")
assert os.path.exists(config_path), "config.js was not created"
with open(config_path, "r") as f:
config_content = f.read()
assert "window.LLAMAINDEX =" in config_content
config_json = json.loads(
config_content.replace("window.LLAMAINDEX = ", "").rstrip(";")
)
assert config_json["APP_TITLE"] == "Dict Config App"
assert config_json["STARTER_QUESTIONS"] == ["Dict Q1", "Dict Q2"]
assert config_json["CHAT_API"] == "/api/chat"
assert config_json["LLAMA_CLOUD_API"] is None
# Clean up
if os.path.exists(".dict_ui"):
shutil.rmtree(".dict_ui")
async def test_component_dir_creation(server: LlamaIndexServer) -> None:
"""
Test if the component directory is created when specified and doesn't exist.
"""
import os
import shutil
test_component_dir = "./test_components"
# Clean up any existing directory
if os.path.exists(test_component_dir):
shutil.rmtree(test_component_dir)
# Create server with component directory
_ = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
ui_config={
"component_dir": test_component_dir,
"include_ui": True,
},
)
# Verify directory was created
assert os.path.exists(test_component_dir), "Component directory was not created"
assert os.path.isdir(test_component_dir), "Component path is not a directory"
# Clean up after test
shutil.rmtree(test_component_dir)
@pytest.mark.asyncio()
async def test_component_router_addition(server: LlamaIndexServer, tmp_path) -> None:
"""
Test if the component router is added when component directory is specified.
"""
test_component_dir = tmp_path / "test_components"
# Create server with component directory
component_server = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
ui_config={
"component_dir": str(test_component_dir),
"include_ui": True,
},
)
# Verify component route exists
component_route_exists = any(
route.path == "/api/components" for route in component_server.routes
)
assert component_route_exists, "Component API route not found in server routes"
@pytest.mark.asyncio()
async def test_ui_config_includes_components_api(
server: LlamaIndexServer, tmp_path
) -> None:
"""
Test if the UI config includes components API when component directory is set.
"""
test_component_dir = tmp_path / "test_components"
# Create server with component directory
component_server = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
ui_config={
"component_dir": str(test_component_dir),
"include_ui": True,
},
)
# Check if components API is in UI config
ui_config = component_server.ui_config
assert "COMPONENTS_API" in ui_config.get_config_content(), (
"Components API not found in UI config"
)
@pytest.mark.asyncio()
async def test_component_router_requires_component_dir(
server: LlamaIndexServer,
) -> None:
"""
Test that adding components router without component_dir raises an error.
"""
server_without_component_dir = LlamaIndexServer(
workflow_factory=_agent_workflow,
verbose=True,
ui_config={
"include_ui": True,
},
)
with pytest.raises(
ValueError, match="component_dir must be specified to add components router"
):
server_without_component_dir.add_components_router()
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "create-llama",
"version": "0.5.1",
"version": "0.5.6",
"description": "Create LlamaIndex-powered apps with one command",
"keywords": [
"rag",
@@ -0,0 +1,409 @@
export default function DeepResearchComponent({ events }) {
// Aggregate events by type and track their state progression
const aggregateEvents = () => {
const retrieveEvents = events.filter((e) => e.event === "retrieve");
const analyzeEvents = events.filter((e) => e.event === "analyze");
const answerEvents = events.filter((e) => e.event === "answer");
// Get the latest state for retrieve and analyze events
const retrieveState =
retrieveEvents.length > 0
? retrieveEvents[retrieveEvents.length - 1].state
: null;
const analyzeState =
analyzeEvents.length > 0
? analyzeEvents[analyzeEvents.length - 1].state
: null;
// Group answer events by their ID
const answerGroups = {};
for (const event of answerEvents) {
if (!event.id) continue;
if (!answerGroups[event.id]) {
answerGroups[event.id] = {
id: event.id,
question: event.question,
answer: event.answer,
states: [],
};
}
const lastState =
answerGroups[event.id].states[answerGroups[event.id].states.length - 1];
if (lastState !== event.state) {
answerGroups[event.id].states.push(event.state);
}
if (event.answer) {
answerGroups[event.id].answer = event.answer;
}
}
return {
retrieveState,
analyzeState,
answerGroups: Object.values(answerGroups),
};
};
const { retrieveState, analyzeState, answerGroups } = aggregateEvents();
// Styles
const styles = {
container: {
maxWidth: "900px",
margin: "0 auto",
padding: "20px",
backgroundColor: "#FFFFFF",
borderRadius: "8px",
boxShadow: "0 2px 8px rgba(0, 0, 0, 0.15)",
},
header: {
fontSize: "24px",
fontWeight: "bold",
textAlign: "center",
marginBottom: "20px",
color: "#333",
},
cardsContainer: {
display: "flex",
gap: "16px",
marginBottom: "30px",
flexWrap: "wrap",
},
card: {
flex: "1",
minWidth: "250px",
backgroundColor: "#FFFFFF",
borderRadius: "8px",
boxShadow: "0 2px 4px rgba(0, 0, 0, 0.1)",
padding: "16px",
border: "1px solid #E5E7EB",
transition: "all 0.3s ease",
},
cardHeader: {
display: "flex",
alignItems: "center",
marginBottom: "12px",
},
cardTitle: {
fontSize: "18px",
fontWeight: "bold",
marginLeft: "8px",
},
cardContent: {
display: "flex",
justifyContent: "space-between",
alignItems: "center",
},
badge: {
padding: "4px 8px",
borderRadius: "9999px",
fontSize: "12px",
fontWeight: "bold",
flexShrink: 0,
},
questionsList: {
marginTop: "30px",
},
questionItem: {
backgroundColor: "#F9FAFB",
border: "1px solid #E5E7EB",
borderRadius: "8px",
marginBottom: "12px",
overflow: "hidden",
},
questionHeader: {
padding: "12px 16px",
cursor: "pointer",
display: "flex",
justifyContent: "space-between",
alignItems: "flex-start",
userSelect: "none",
},
questionTitle: {
fontWeight: "medium",
marginLeft: "12px",
},
answerContainer: {
padding: "0",
backgroundColor: "#F3F4F6",
overflow: "hidden",
maxHeight: "0",
transition: "max-height 0.3s ease-out",
whiteSpace: "pre-line",
fontSize: "13px",
},
answerContent: {
padding: "16px",
},
answerContainerExpanded: {
maxHeight: "1000px", // Adjust based on content
},
arrow: {
marginLeft: "8px",
transition: "transform 0.3s ease",
display: "inline-block",
},
loadingContainer: {
display: "flex",
justifyContent: "center",
alignItems: "center",
padding: "24px",
color: "#6B7280",
},
stateIconContainer: {
display: "flex",
alignItems: "center",
gap: "8px",
},
};
// Helper function to get color for a state
const getStateColor = (state) => {
switch (state) {
case "inprogress":
return "#FCD34D";
case "done":
return "#34D399";
case "pending":
return "#9CA3AF";
default:
return "#D1D5DB";
}
};
// Helper function to get badge styles based on state
const getBadgeStyles = (state) => {
const colors = {
inprogress: { background: "#FEF3C7", color: "#92400E" },
done: { background: "#D1FAE5", color: "#065F46" },
pending: { background: "#F3F4F6", color: "#1F2937" },
};
const stateColors = colors[state] || colors.pending;
return {
...styles.badge,
backgroundColor: stateColors.background,
color: stateColors.color,
};
};
// Helper function to render state icon
const renderStateIcon = (state) => {
const color = getStateColor(state);
if (state === "inprogress") {
return (
<div style={{ color, animation: "spin 1s linear infinite" }}></div>
);
} else if (state === "done") {
return <div style={{ color }}></div>;
} else if (state === "pending") {
return <div style={{ color }}></div>;
}
return <div style={{ color }}>?</div>;
};
// State for toggling question answers
const [expandedQuestions, setExpandedQuestions] = React.useState({});
const toggleQuestion = (questionId) => {
setExpandedQuestions((prev) => ({
...prev,
[questionId]: !prev[questionId],
}));
};
return (
<div style={styles.container}>
<h1 style={styles.header}>Research Progress</h1>
{/* Status Cards */}
<div style={styles.cardsContainer}>
{/* Retrieve Card */}
<div
style={{
...styles.card,
borderColor:
retrieveState === "done"
? "#A7F3D0"
: retrieveState === "inprogress"
? "#FDE68A"
: "#E5E7EB",
}}
>
<div style={styles.cardHeader}>
<span style={{ color: "#8B5CF6" }}>🔍</span>
<div style={styles.cardTitle}>Data Retrieval</div>
</div>
<div style={styles.cardContent}>
<span style={{ fontSize: "14px", color: "#6B7280" }}>Status:</span>
<div style={styles.stateIconContainer}>
{retrieveState && (
<span style={getBadgeStyles(retrieveState)}>
{retrieveState === "inprogress"
? "In Progress"
: retrieveState === "done"
? "Completed"
: "Pending"}
</span>
)}
{renderStateIcon(retrieveState)}
</div>
</div>
</div>
{/* Analyze Card */}
<div
style={{
...styles.card,
borderColor:
analyzeState === "done"
? "#A7F3D0"
: analyzeState === "inprogress"
? "#FDE68A"
: "#E5E7EB",
}}
>
<div style={styles.cardHeader}>
<span style={{ color: "#06B6D4" }}>📊</span>
<div style={styles.cardTitle}>Data Analysis</div>
</div>
<div style={styles.cardContent}>
<span style={{ fontSize: "14px", color: "#6B7280" }}>Status:</span>
<div style={styles.stateIconContainer}>
{analyzeState && (
<span style={getBadgeStyles(analyzeState)}>
{analyzeState === "inprogress"
? "In Progress"
: analyzeState === "done"
? "Completed"
: "Pending"}
</span>
)}
{renderStateIcon(analyzeState)}
</div>
</div>
</div>
{/* Questions Card */}
<div style={styles.card}>
<div style={styles.cardHeader}>
<span style={{ color: "#10B981" }}>💬</span>
<div style={styles.cardTitle}>Questions</div>
</div>
<div style={styles.cardContent}>
<span style={{ fontSize: "14px", color: "#6B7280" }}>Status:</span>
<div style={{ display: "flex", alignItems: "center", gap: "8px" }}>
<span
style={{
...styles.badge,
backgroundColor: "#F3F4F6",
color: "#1F2937",
}}
>
{answerGroups.length} Questions
</span>
<span
style={{
fontSize: "14px",
fontWeight: "500",
color: "#059669",
}}
>
{answerGroups.filter((g) => g.states.includes("done")).length}{" "}
Answered
</span>
</div>
</div>
</div>
</div>
{/* Questions List */}
{answerGroups.length > 0 && (
<div style={styles.questionsList}>
<h2
style={{
fontSize: "20px",
fontWeight: "600",
marginBottom: "16px",
}}
>
Research Questions
</h2>
{answerGroups.map((group, index) => {
const latestState = group.states[group.states.length - 1];
const questionId = group.id || `question-${index}`;
const isExpanded = expandedQuestions[questionId];
const answerWithoutCitation = group.answer?.replace(
/\[citation:[a-f0-9-]+\]/g,
"",
);
return (
<div key={questionId} style={styles.questionItem}>
<div
style={styles.questionHeader}
onClick={() => toggleQuestion(questionId)}
>
<div
style={{
display: "flex",
alignItems: "flex-start",
gap: "12px",
fontSize: "14px",
}}
>
{renderStateIcon(latestState)}
<span style={styles.questionTitle}>{group.question}</span>
<span
style={{
...styles.arrow,
transform: isExpanded
? "rotate(180deg)"
: "rotate(0deg)",
}}
>
</span>
</div>
<span style={getBadgeStyles(latestState)}>
{latestState === "inprogress"
? "In Progress"
: latestState === "done"
? "Answered"
: "Pending"}
</span>
</div>
<div
style={{
...styles.answerContainer,
...(isExpanded ? styles.answerContainerExpanded : {}),
}}
>
{answerWithoutCitation ? (
<div style={styles.answerContent}>
{answerWithoutCitation}
</div>
) : (
<div
style={{
...styles.loadingContainer,
...styles.answerContent,
}}
>
<span style={{ marginLeft: "8px" }}>
Generating answer...
</span>
</div>
)}
</div>
</div>
);
})}
</div>
)}
</div>
);
}
@@ -49,6 +49,16 @@ curl --location 'localhost:8000/api/chat' \
--data '{ "messages": [{ "role": "user", "content": "Create a report comparing the finances of Apple and Tesla" }] }'
```
## Customize the UI
To customize the UI, you can start by modifying the [./components/ui_event.jsx](./components/ui_event.jsx) file.
You can also generate a new code for the workflow using LLM by running the following command:
```
poetry run generate:ui
```
## Learn More
To learn more about LlamaIndex, take a look at the following resources:
@@ -23,8 +23,7 @@ from llama_index.core.workflow import (
Workflow,
step,
)
from llama_index.server.api.models import SourceNodesEvent
from llama_index.server.api.models import ChatRequest
from llama_index.server.api.models import ChatRequest, SourceNodesEvent, UIEvent
from pydantic import BaseModel, Field
logger = logging.getLogger("uvicorn")
@@ -66,20 +65,29 @@ class ReportEvent(Event):
# Events that are streamed to the frontend and rendered there
class DeepResearchEventData(BaseModel):
event: Literal["retrieve", "analyze", "answer"]
state: Literal["pending", "inprogress", "done", "error"]
id: Optional[str] = None
question: Optional[str] = None
answer: Optional[str] = None
class UIEventData(BaseModel):
"""
Events for DeepResearch workflow which has 3 main stages:
- Retrieve: Retrieve information from the knowledge base.
- Analyze: Analyze the retrieved information and provide list of questions for answering.
- Answer: Answering the provided questions. There are multiple answer events, each with its own id that is used to display the answer for a particular question.
"""
class DataEvent(Event):
type: Literal["deep_research_event"]
data: DeepResearchEventData
def to_response(self):
return self.model_dump()
id: Optional[str] = Field(default=None, description="The id of the event")
event: Literal["retrieve", "analyze", "answer"] = Field(
default="retrieve", description="The event type"
)
state: Literal["pending", "inprogress", "done", "error"] = Field(
default="pending", description="The state of the event"
)
question: Optional[str] = Field(
default=None,
description="Used by answer event to display the question",
)
answer: Optional[str] = Field(
default=None,
description="Used by answer event to display the answer of the question",
)
class DeepResearchWorkflow(Workflow):
@@ -137,12 +145,12 @@ class DeepResearchWorkflow(Workflow):
]
)
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "retrieve",
"state": "inprogress",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="retrieve",
state="inprogress",
),
)
)
retriever = self.index.as_retriever(
@@ -151,12 +159,12 @@ class DeepResearchWorkflow(Workflow):
nodes = retriever.retrieve(self.user_request)
self.context_nodes.extend(nodes) # type: ignore
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "retrieve",
"state": "done",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="retrieve",
state="done",
),
)
)
# Send source nodes to the stream
@@ -177,12 +185,12 @@ class DeepResearchWorkflow(Workflow):
"""
logger.info("Analyzing the retrieved information")
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "analyze",
"state": "inprogress",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="analyze",
state="inprogress",
),
)
)
total_questions = await ctx.get("total_questions")
@@ -194,12 +202,12 @@ class DeepResearchWorkflow(Workflow):
)
if res.decision == "cancel":
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "analyze",
"state": "done",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="analyze",
state="done",
),
)
)
return StopEvent(
@@ -210,12 +218,12 @@ class DeepResearchWorkflow(Workflow):
# It's a LLM hallucination.
if total_questions == 0:
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "analyze",
"state": "done",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="analyze",
state="done",
),
)
)
return StopEvent(
@@ -245,15 +253,15 @@ class DeepResearchWorkflow(Workflow):
for question in res.research_questions:
question_id = str(uuid.uuid4())
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "answer",
"state": "pending",
"id": question_id,
"question": question,
"answer": None,
},
UIEvent(
type="ui_event",
data=UIEventData(
event="answer",
state="pending",
id=question_id,
question=question,
answer=None,
),
)
)
ctx.send_event(
@@ -264,12 +272,12 @@ class DeepResearchWorkflow(Workflow):
)
)
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "analyze",
"state": "done",
},
UIEvent(
type="ui_event",
data=UIEventData(
event="analyze",
state="done",
),
)
)
return None
@@ -280,14 +288,14 @@ class DeepResearchWorkflow(Workflow):
Answer the question
"""
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "answer",
"state": "inprogress",
"id": ev.question_id,
"question": ev.question,
},
UIEvent(
type="ui_event",
data=UIEventData(
event="answer",
state="inprogress",
id=ev.question_id,
question=ev.question,
),
)
)
try:
@@ -299,15 +307,15 @@ class DeepResearchWorkflow(Workflow):
logger.error(f"Error answering question {ev.question}: {e}")
answer = f"Got error when answering the question: {ev.question}"
ctx.write_event_to_stream(
DataEvent(
type="deep_research_event",
data={
"event": "answer",
"state": "done",
"id": ev.question_id,
"question": ev.question,
"answer": answer,
},
UIEvent(
type="ui_event",
data=UIEventData(
event="answer",
state="done",
id=ev.question_id,
question=ev.question,
answer=answer,
),
)
)
@@ -29,6 +29,10 @@ Open [http://localhost:3000](http://localhost:3000) with your browser to see the
You can configure [LLM model](https://ts.llamaindex.ai/docs/llamaindex/modules/llms) and [embedding model](https://ts.llamaindex.ai/docs/llamaindex/modules/embeddings) in the [settings file](src/app/settings.ts).
## Custom UI Components
For Deep Research, we have a custom component located in `components/deep_research_event.jsx`. This is used to display the results of the deep research workflow in a more user-friendly way
## Use Case
We have prepared an [example workflow](./src/app/workflow.ts) for the Deep Research use case, where you can request a detailed answer about the example documents in the [./data](./data) directory.
@@ -1,8 +1,4 @@
import {
DeepResearchEvent,
toSourceEvent,
toStreamGenerator,
} from "@llamaindex/server";
import { toSourceEvent, toStreamGenerator } from "@llamaindex/server";
import {
AgentInputData,
AgentWorkflowContext,
@@ -123,6 +119,19 @@ class ResearchEvent extends WorkflowEvent<ResearchQuestion[]> {}
class ReportEvent extends WorkflowEvent<{}> {}
class StopEvent extends StopEventBase<AsyncGenerator<ChatResponseChunk>> {}
type DeepResearchEventData = {
event: "retrieve" | "analyze" | "answer";
state: "pending" | "inprogress" | "done" | "error";
id?: string;
question?: string;
answer?: string;
};
class DeepResearchEvent extends WorkflowEvent<{
type: "ui_event";
data: DeepResearchEventData;
}> {}
// workflow definition
class DeepResearchWorkflow extends Workflow<
AgentWorkflowContext,
@@ -192,7 +201,7 @@ class DeepResearchWorkflow extends Workflow<
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "retrieve", state: "inprogress" },
}),
);
@@ -203,7 +212,7 @@ class DeepResearchWorkflow extends Workflow<
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "retrieve", state: "done" },
}),
);
@@ -219,7 +228,7 @@ class DeepResearchWorkflow extends Workflow<
): Promise<ResearchEvent | ReportEvent | StopEvent> => {
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "analyze", state: "inprogress" },
}),
);
@@ -231,7 +240,7 @@ class DeepResearchWorkflow extends Workflow<
if (decision === "cancel") {
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "analyze", state: "done" },
}),
);
@@ -254,7 +263,7 @@ class DeepResearchWorkflow extends Workflow<
researchQuestions.forEach(({ questionId: id, question }) => {
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "answer", state: "pending", id, question },
}),
);
@@ -271,7 +280,7 @@ class DeepResearchWorkflow extends Workflow<
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "analyze", state: "done" },
}),
);
@@ -290,7 +299,7 @@ class DeepResearchWorkflow extends Workflow<
researchQuestions.map(async ({ questionId: id, question }) => {
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "answer", state: "inprogress", id, question },
}),
);
@@ -299,7 +308,7 @@ class DeepResearchWorkflow extends Workflow<
ctx.sendEvent(
new DeepResearchEvent({
type: "deep_research_event",
type: "ui_event",
data: { event: "answer", state: "done", id, question, answer },
}),
);
@@ -1,19 +1,24 @@
import logging
import os
from app.index import STORAGE_DIR
from app.settings import init_settings
from dotenv import load_dotenv
from llama_index.core.indices import (
VectorStoreIndex,
)
from llama_index.core.readers import SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
def generate_datasource():
def generate_index():
"""
Index the documents in the data directory.
"""
from app.index import STORAGE_DIR
from app.settings import init_settings
from llama_index.core.indices import (
VectorStoreIndex,
)
from llama_index.core.readers import SimpleDirectoryReader
load_dotenv()
init_settings()
@@ -31,3 +36,28 @@ def generate_datasource():
# store it for later
index.storage_context.persist(STORAGE_DIR)
logger.info(f"Finished creating new index. Stored in {STORAGE_DIR}")
def generate_ui_for_workflow():
"""
Generate UI for UIEventData event in app/workflow.py
"""
import asyncio
from main import COMPONENT_DIR
# To generate UI components for additional event types,
# import the corresponding data model (e.g., MyCustomEventData)
# and run the generate_ui_for_workflow function with the imported model.
# Make sure the output filename of the generated UI component matches the event type (here `ui_event`)
try:
from app.workflow import UIEventData
except ImportError:
raise ImportError("Couldn't generate UI component for the current workflow.")
from llama_index.server.gen_ui import generate_event_component
# works also well with Claude 3.7 Sonnet or Gemini Pro 2.5
llm = OpenAI(model="gpt-4.1")
code = asyncio.run(generate_event_component(event_cls=UIEventData, llm=llm))
with open(f"{COMPONENT_DIR}/ui_event.jsx", "w") as f:
f.write(code)
@@ -5,16 +5,23 @@ import subprocess
from app.settings import init_settings
from app.workflow import create_workflow
from dotenv import load_dotenv
from llama_index.server import LlamaIndexServer
from llama_index.server import LlamaIndexServer, UIConfig
logger = logging.getLogger("uvicorn")
# A path to a directory where the customized UI code is stored
COMPONENT_DIR = "components"
def create_app():
env = os.environ.get("APP_ENV")
app = LlamaIndexServer(
workflow_factory=create_workflow, # A factory function that creates a new workflow for each request
ui_config=UIConfig(
component_dir=COMPONENT_DIR,
app_title="Chat App",
),
env=env,
logger=logger,
)
@@ -7,7 +7,9 @@ authors = ["Marcus Schiesser <mail@marcusschiesser.de>"]
readme = "README.md"
[tool.poetry.scripts]
generate = "generate:generate_datasource"
"generate" = "generate:generate_index"
"generate:index" = "generate:generate_index"
"generate:ui" = "generate:generate_ui_for_workflow"
dev = "main:run('dev')"
prod = "main:run('prod')"
@@ -16,8 +18,8 @@ python = ">=3.11,<3.14"
python-dotenv = "^1.0.0"
pydantic = "<2.10"
aiostream = "^0.5.2"
llama-index-core = "0.12.25"
llama-index-server = "^0.1.7"
llama-index-core = "^0.12.28"
llama-index-server = "^0.1.13"
[tool.poetry.group.dev.dependencies]
mypy = "^1.8.0"
@@ -9,10 +9,10 @@
"dependencies": {
"@llamaindex/openai": "0.2.0",
"@llamaindex/readers": "^2.0.0",
"@llamaindex/server": "0.0.6",
"@llamaindex/server": "0.1.3",
"@llamaindex/tools": "0.0.4",
"dotenv": "^16.4.7",
"llamaindex": "0.9.15",
"llamaindex": "0.9.17",
"zod": "^3.23.8"
},
"devDependencies": {
@@ -7,5 +7,8 @@ initSettings();
new LlamaIndexServer({
workflow: workflowFactory,
appTitle: "LlamaIndex App",
uiConfig: {
appTitle: "LlamaIndex App",
componentsDir: "components",
},
}).start();