Commit Graph

77 Commits

Author SHA1 Message Date
Adrian Lyjak e9bf27bc11 perf(ci): tune buildx GHA cache for docker publish jobs (#547)
The publish workflow's docker-build step for the operator image was
spending ~2 minutes of every run exporting cache to GHA (per build
log: `sending cache export 88.3s done`, total step 3m37s). The
`golang:1.24` base + builder-stage layers were being uploaded to GHA
cache on every push because `cache-to` was hardcoded to `mode=max`.

Changes:
- Add `cacheMode` to `DockerConfig` (defaults to `max` — the Python
  images still benefit from caching every intermediate stage).
- Plumb it through `DockerBuildAction` and into the `--cache-to`
  flag in `_execute_docker_build`.
- Always emit `compression=zstd` and `ignore-error=true` on the
  cache-to directive: smaller blobs to upload, and a flaky cache
  push no longer fails a build whose image is already on the registry.
- Set `"cacheMode": "min"` on `operator/package.json`. The operator
  is a tiny static Go binary in distroless; caching the golang base
  + module download layers costs more upload time than the rebuild
  saves.

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-15 15:09:50 -04:00
Adrian Lyjak a5f1140231 fix: reset working tree after changesets/action before planning (#504) 2026-04-09 11:11:30 -04:00
Adrian Lyjak 42bc68543d Optimize publish action (#489) 2026-04-07 16:59:55 -04:00
Adrian Lyjak 62eee92c82 Add llama agents cli, serving, and deployment packages (#467) 2026-04-06 11:46:05 -04:00
Adrian Lyjak 49d1c21e93 Add test count to dev test runner (#354) 2026-02-27 17:02:02 -05:00
Adrian Lyjak 7bc7750043 Add semver ↔ PEP 440 version format conversion utilities (#345) 2026-02-09 13:27:16 -05:00
Adrian Lyjak 0d8f5a7b27 fix: improve dev CLI pytest robustness (#320) 2026-02-02 19:02:44 -05:00
Adrian Lyjak 5ded4b40fa chore: CI/testing improvements for parallel test execution (#317)
* chore: CI/testing improvements for parallel test execution

- Add workflows-dev pytest command with parallel test runner
- Add pytest-xdist for parallel test execution (-nauto)
- Add timeouts to prevent test hangs (--timeout=10/60/120)
- Use module-scoped async fixtures for faster tests
- Remove mypy in favor of basedpyright/ty type checking
- Add test-docker CI job for integration tests with testcontainers
- Exclude integration tests from Python 3.9/3.14 in CI matrix
2026-02-02 13:27:10 -05:00
Adrian Lyjak 4d93f89730 Update workflows for better mono-repo build support (#202)
* Make llama-index-workflows into a sub package

* Attempting to add changesets (#207)

---------

Co-authored-by: Clelia (Astra) Bertelli <133636879+AstraBert@users.noreply.github.com>
2025-11-17 14:18:13 -05:00
Adrian Lyjak babb1abe13 Fix reconnect (#199)
* Fix reconnect

* Bump version to 2.11.1
2025-11-07 09:56:29 -05:00
Adrian Lyjak 7b97d338d5 Add more error logging, and add handler and run dispatcher context (#194) 2025-11-05 23:02:57 -05:00
Adrian Lyjak 2639af513b Improve workflow configuration error message (#182) 2025-11-05 21:29:25 -05:00
Adrian Lyjak 988149b6c6 handle non picklable requirements and events (#187) 2025-11-03 11:08:58 -05:00
Adrian Lyjak 776efe757e validate that workflow is not cleared until events are consumed (#186)
* validate that workflow is not cleared until events are consumed

* version bump

* move completion check to function located with status union
2025-11-01 21:16:58 -04:00
Adrian Lyjak e50ce4bab5 Improve stop event string representation (#184) 2025-11-01 20:35:28 -04:00
Adrian Lyjak f601e6bb4f Fixup sqlite persistence (#183) 2025-11-01 20:35:09 -04:00
Adrian Lyjak 43274071bd Add persistence filters to list handlers (#174) 2025-10-30 12:18:02 -04:00
Terry Zhao 3d512ac8e2 fix bugs (#178) 2025-10-30 08:55:52 -04:00
Adrian Lyjak 0ab91bdcfa Return fully qualified type from handler result (#171) 2025-10-29 21:23:13 -04:00
Adrian Lyjak 4f98e80fe4 fix: Fix for unhashable workflow subclasses (#177) 2025-10-29 14:18:18 -04:00
Adrian Lyjak 18d9bb9669 Refactor context to (eventually) support runtime plugins (#127) 2025-10-28 14:32:41 -04:00
Clelia (Astra) Bertelli a0f004c25d Enhancements for workflow server docs and client cleanup (#154)
* docs: enhanced workflow server docs; chore: add cancel handler method to client

* ci: lint

* fix: correct cancel endpoint and add test

* chore: pr review suggestions
2025-10-23 17:32:07 +02:00
Terry Zhao 78ef017f2a handlers list improvement (#153)
* fix handlers list

* fix lint

* Add sqlite migrations (#155)

* Add sqlite migrations

* remove inline import

* move memory store to sever/

* addd store

* resolve comment

* fix lint

---------

Co-authored-by: Adrian Lyjak <adrianlyjak@gmail.com>
2025-10-22 14:20:51 -07:00
Adrian Lyjak 8a0a8aabea Normalize server/client event schema (#152) 2025-10-20 13:04:49 -04:00
Adrian Lyjak 24acc320be Fix workflow client send event (#150) 2025-10-17 11:29:34 -04:00
Terry Zhao 0f94b0292b automation for update debugger index script & css (#147)
* automation

* fix test

* fix lint
2025-10-17 00:50:38 -07:00
Logan c241d5f710 add proper serialization to default stop event (#144) 2025-10-15 16:32:37 -06:00
Clelia (Astra) Bertelli 5607489f9e Add python workflow client (#109) 2025-10-10 16:23:06 -04:00
Terry Zhao f05f51985d add event envelope (#133)
* add event envolope

* add missing file

* format

* resolve comments

* lint

* fix nit
2025-10-10 20:12:42 +00:00
Logan 2de91c74ce Simplify schema for send event to handler in WorkflowServer (#129) 2025-10-08 15:56:36 -06:00
Logan 85d39858e4 Expose workflow events in server and workflow class (#126) 2025-10-08 09:04:18 -06:00
Adrian Lyjak 76af884ec9 Don't skip validation if start event is empty or missing (#123) 2025-10-06 14:44:23 -04:00
Adrian Lyjak 686293f526 fix: Add single-consumer lock for event streams (#118) 2025-10-01 16:25:31 -04:00
Adrian Lyjak d9aef2bbae Add WorkflowServer API to stop workflow handler and cancel tasks (#117) 2025-09-30 23:02:59 -04:00
Adrian Lyjak d5f471adfa Prevent workflow server crash on invalid reload (#116)
* Fix: Handle errors during workflow resume and context restoration

Co-authored-by: adrian <adrian@runllama.ai>

* Refactor server startup to handle invalid contexts gracefully

Co-authored-by: adrian <adrian@runllama.ai>

* Fix: Improve invalid context test case

Co-authored-by: adrian <adrian@runllama.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-30 13:22:24 -04:00
Adrian Lyjak fae7626a39 Filter internal events in workflow consumer (#115)
* Add option to include internal workflow events in server stream

Co-authored-by: adrian <adrian@runllama.ai>

* Refactor: Rename 'internal' query param to 'include_internal'

Co-authored-by: adrian <adrian@runllama.ai>

* Remove unnecessary dependencies from pre-commit config

Co-authored-by: adrian <adrian@runllama.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-30 13:22:08 -04:00
Logan 8426da813b able to pass in previous handler ID to continue runs (#113) 2025-09-25 17:52:04 -06:00
Adrian Lyjak 62150fcfef Fix result serialization, and externalize debugger (#112) 2025-09-25 18:57:36 -04:00
Terry Zhao 337e84c994 server: flattened handler metadata in responses, shared OpenAPI components, and test updates (#110)
* server: return ISO 8601 timestamps (Z) for started_at/updated_at/completed_at and update OpenAPI to date-time format

* server: format timestamps with strftime('%Y-%m-%dT%H:%M:%SZ') for canonical ISO-8601Z output

* tests(server): add datetime progression test for handler started_at/updated_at/completed_at

* fix lint
2025-09-23 15:49:04 -07:00
Terry Zhao cf6609436b fix 2025-09-23 15:40:42 -07:00
Adrian Lyjak 2d3d3a4f72 feat: switch to SSE by default (#107)
* feat: switch to SSE by default

* Remove commented out comment
2025-09-22 16:51:23 -04:00
Clelia (Astra) Bertelli 256d4b3695 chore: migrating tests (pt.2) (#103)
* feat: migrating tests pt. 2

* fix: make mypy happy

* feat: allow passing context to test runner
2025-09-22 10:43:12 +02:00
Adrian Lyjak f0e9d2d7d4 Add better error handling and lifecycle handling to workflow server (#101) 2025-09-17 14:08:08 -04:00
Clelia (Astra) Bertelli d1e0a7dd6d feat: first attempt at test migration to WorkflowTestRunner (#93)
* feat: first attempt at test migration to WorkflowTestRunner

* chore: use StartEvent() as default for start_event in test runner
2025-09-17 12:57:54 +02:00
Logan 3db3847186 fix wait_for_event when you have a custom context state definition (#102)
* Do not write to store on users behalf

* fix rountrip hitl serialization

* remove finally clause
2025-09-17 11:05:20 +02:00
Adrian Lyjak bbaf147c08 simple workflow server persistence (#92)
* Add basic pluggable single server/worker persistence with sqlite option

* complete event

* tweak tests, comments

* Code review feedback

* Add xdist

* not that context
2025-09-12 13:35:16 -06:00
Adrian Lyjak 48d973391a feat: support more start event input formats in server API (#94)
* feat: support more start event input formats, so client doesnt need knowledge of python path

* remove forgotten comments
2025-09-12 14:35:26 -04:00
Clelia (Astra) Bertelli 67d14d4542 feat: add testing utils (#84)
* feat: add testing utils

* fix: make python 3.9 happy

* chore: removed run_test method on Workflow and send_test_event on WorkflowTestRunner, rename run_and_collect to run

* chore: make the test runner an async context manager again

* chore: removing the context manager
2025-09-11 16:42:01 +02:00
Terry Zhao f15bfa10cc feat: Add POST /events/{handler_id} endpoint and /handlers endpoint (#89)
* feat: Add POST /events/{handler_id} endpoint and /tasks endpoint

- Add GET /tasks endpoint to list running workflow tasks
- Add POST /events/{handler_id} endpoint to send events to running workflows
- Enable external event injection into workflow contexts
- Add InteractiveWorkflow test fixture for testing event handling
- Add comprehensive tests for both endpoints

This allows developers to:
1. Monitor running workflows via the /tasks endpoint
2. Send events to running workflows for interactive use cases
3. Build event-driven integrations with external systems

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix lint

* refactor: Update /tasks endpoint to /handlers with enhanced status tracking

- Renamed /tasks endpoint to /handlers for better naming consistency
- Enhanced handler status tracking (running, completed, failed)
- Added result and error fields to handler response
- Fixed safe access to handler.result() and handler.exception() methods
- Added comprehensive tests for all handler states

The /handlers endpoint now provides complete visibility into workflow execution state.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* update comments

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-10 14:18:05 -07:00
Terry Zhao 91d041f2fd feat: Add TypeScript SDK generation and publishing pipeline (#83)
* inital commit

* feat: Add TypeScript SDK generation and publishing pipeline

- Add @hey-api/openapi-ts based SDK generation from Starlette OpenAPI schema
- Create unified release workflow with separate jobs for Python and TypeScript
- Add version validation to ensure Git tags match pyproject.toml
- Configure SDK to use @llamaindex/workflows-client package name
- Add CI workflow to validate SDK generation on every PR
- Use pnpm as package manager for Node.js operations
- Add pyyaml dependency for OpenAPI schema generation

The SDK is treated as a build artifact, generated from the Python server's
OpenAPI specification. Version synchronization is maintained by using
pyproject.toml as the single source of truth.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix lint

* fix

* rebase main

* fix

* fix

* fix ci

* refactor: Simplify to only publish OpenAPI spec as release artifact

- Remove TypeScript SDK generation infrastructure
- Add OpenAPI spec generation to release workflow
- Add version validation to ensure tag matches pyproject.toml
- Add trigger to notify llama-ui for SDK generation
- Keep workflow changes minimal from original

The SDK generation will now be handled in the llama-ui repository,
which can fetch the OpenAPI spec from releases and generate the SDK.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add smart SDK update triggering with change type detection

- Add workflow_dispatch inputs for manual server change type and description
- Create script to auto-detect change type from semantic version tags
- Add comprehensive tests using pytest-style functions with mock data
- Only trigger SDK update when there are server/API changes
- Pass change type and description to llama-ui for proper versioning
- Use tomllib for proper TOML parsing in validate_version.py

For manual triggers: developers can specify if there are API changes
For tag pushes: automatically detects patch/minor/major from version bump

Tests use temporary directories and mock data for complete isolation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix lint

* fix: Replace tomllib with regex parsing for Python 3.9+ compatibility

- Remove tomllib dependency which is only available in Python 3.11+
- Use regex to parse version from pyproject.toml for broader Python version support
- Add test fixture cleanup to reduce event loop warnings in async tests

* fix comment

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-08 16:35:38 -07:00