deepset-ai/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-10-12 02:12:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: Move HITL to underneath the Hooks module (#11974)

Browse Source
This commit is contained in:
Sebastian Husch Lee
2026-07-13 09:15:40 +00:00
committed by GitHub
parent 6b379741ea
commit 86d46976c1
24 changed files with 95 additions and 554 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MIGRATION.md
+3 -1
View File
@@ -639,6 +639,8 @@ agent = Agent(
**Why:** Confirmation was a one-off, before-tool interception bolted onto the Agent. Hooks generalize that seam, so HITL becomes one application of a single, uniform extension point instead of a parallel concept with its own serialization and run plumbing.
The Human-in-the-Loop module has also moved from `haystack.human_in_the_loop` to `haystack.hooks.human_in_the_loop`, so that it lives alongside the other built-in hooks (such as tool result offloading). Update your imports to the new location.
**How to migrate:**
Before (v2.x):
@@ -661,7 +663,7 @@ agent.run(messages=[...], confirmation_strategy_context={"websocket": ws})
After (v3.0):
```python
from haystack.components.agents import Agent
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
BlockingConfirmationStrategy,
AlwaysAskPolicy,
ConfirmationHook,
docs-website/docs/pipeline-components/agents-1/hooks.mdx
+3 -3
View File
@@ -196,7 +196,7 @@ print(result["last_message"].text)
## Ready-made hooks
Haystack ships two ready-made hooks:
Haystack ships two ready-made hooks, each in its own submodule of `haystack.hooks`:
- `ConfirmationHook`: A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx).
- `ToolResultOffloadHook`: An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. See [Tool Result Offloading](./tool-result-offloading.mdx).
- `ConfirmationHook` (from `haystack.hooks.human_in_the_loop`): A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx).
- `ToolResultOffloadHook` (from `haystack.hooks.tool_result_offloading`): An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. See [Tool Result Offloading](./tool-result-offloading.mdx).
docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx
+14 -11
View File
@@ -17,8 +17,8 @@ This is useful for high-stakes operations - such as sending emails, modifying da
| --- | --- |
| **Configured on** | The [`Agent`](./agent.mdx) component, as a `ConfirmationHook` registered under the `before_tool` [hook point](./hooks.mdx) |
| **Key classes** | `ConfirmationHook`, `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` |
| **Import path** | `haystack.human_in_the_loop` |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/human_in_the_loop/ |
| **Import path** | `haystack.hooks.human_in_the_loop` |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/human_in_the_loop/ |
| **Package name** | `haystack-ai` |
</div>
@@ -56,7 +56,7 @@ from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
@@ -118,7 +118,7 @@ pip install rich
```
```python
from haystack.human_in_the_loop import RichConsoleUI
from haystack.hooks.human_in_the_loop import RichConsoleUI
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
@@ -202,10 +202,13 @@ Policies control *when* the human is asked.
### Custom policy
You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.human_in_the_loop.types`:
You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.hooks.human_in_the_loop.types`:
```python
from haystack.human_in_the_loop.types import ConfirmationPolicy, ConfirmationUIResult
from haystack.hooks.human_in_the_loop.types import (
ConfirmationPolicy,
ConfirmationUIResult,
)
from typing import Any
@@ -227,8 +230,8 @@ It is called after the user responds and receives the full `ConfirmationUIResult
The following policy asks once per tool name and skips re-asking for any tool the user has already confirmed:
```python
from haystack.human_in_the_loop.types import ConfirmationPolicy
from haystack.human_in_the_loop import ConfirmationUIResult
from haystack.hooks.human_in_the_loop.types import ConfirmationPolicy
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from typing import Any
@@ -295,11 +298,11 @@ This is a good reference if you need non-blocking HITL in a web or server enviro
## Custom UI
Implement `ConfirmationUI` from `haystack.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
Implement `ConfirmationUI` from `haystack.hooks.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
```python
from haystack.human_in_the_loop.types import ConfirmationUI
from haystack.human_in_the_loop import ConfirmationUIResult
from haystack.hooks.human_in_the_loop.types import ConfirmationUI
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from typing import Any
docs-website/reference/haystack-api/human_in_the_loop_api.md
-471
View File
@@ -1,471 +0,0 @@
---
title: "Human-in-the-Loop"
id: human-in-the-loop-api
description: "Abstractions for integrating human feedback and interaction into Agent workflows."
slug: "/human-in-the-loop-api"
---
## dataclasses
### ConfirmationUIResult
Result of the confirmation UI interaction.
**Parameters:**
- **action** (<code>str</code>) – The action taken by the user such as "confirm", "reject", or "modify".
This action type is not enforced to allow for custom actions to be implemented.
- **feedback** (<code>str | None</code>) – Optional feedback message from the user. For example, if the user rejects the tool execution,
they might provide a reason for the rejection.
- **new_tool_params** (<code>dict\[str, Any\] | None</code>) – Optional set of new parameters for the tool. For example, if the user chooses to modify the tool parameters,
they can provide a new set of parameters here.
### ToolExecutionDecision
Decision made regarding tool execution.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **execute** (<code>bool</code>) – A boolean indicating whether to execute the tool with the provided parameters.
- **tool_call_id** (<code>str | None</code>) – Optional unique identifier for the tool call. This can be used to track and correlate the decision with a
specific tool invocation.
- **feedback** (<code>str | None</code>) – Optional feedback message.
For example, if the tool execution is rejected, this can contain the reason. Or if the tool parameters were
modified, this can contain the modification details.
- **final_tool_params** (<code>dict\[str, Any\] | None</code>) – Optional final parameters for the tool if execution is confirmed or modified.
#### to_dict
```python
to_dict() -> dict[str, Any]
```
Convert the ToolExecutionDecision to a dictionary representation.
**Returns:**
- <code>dict\[str, Any\]</code> – A dictionary containing the tool execution decision details.
#### from_dict
```python
from_dict(data: dict[str, Any]) -> ToolExecutionDecision
```
Populate the ToolExecutionDecision from a dictionary representation.
**Parameters:**
- **data** (<code>dict\[str, Any\]</code>) – A dictionary containing the tool execution decision details.
**Returns:**
- <code>ToolExecutionDecision</code> – An instance of ToolExecutionDecision.
## hooks
### ConfirmationHook
A `before_tool` Agent hook that applies Human-in-the-Loop confirmation strategies to pending tool calls.
Register it on an `Agent` to confirm, modify, or reject tool calls before they run:
```python
from haystack.components.agents import Agent
from haystack.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
NeverAskPolicy,
RichConsoleUI,
SimpleConsoleUI,
)
hook = ConfirmationHook(
confirmation_strategies={
"my_tool": BlockingConfirmationStrategy(
confirmation_policy=NeverAskPolicy(), confirmation_ui=SimpleConsoleUI()
)
}
)
agent = Agent(chat_generator=..., tools=[...], hooks={"before_tool": [hook]})
```
A key may be a single tool name, a tuple of tool names sharing one strategy, or the wildcard `"*"` which applies
to any tool without a more specific entry. More specific keys win, so you can set a default for all tools and
override individual ones:
```python
hook = ConfirmationHook(
confirmation_strategies={
"delete_file": BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(), confirmation_ui=RichConsoleUI()
),
"*": BlockingConfirmationStrategy(
confirmation_policy=NeverAskPolicy(), confirmation_ui=SimpleConsoleUI()
),
}
)
```
Request-scoped resources for the strategies (e.g. a WebSocket or queue) are passed per run via the Agent's
`hook_context` argument (`agent.run(messages=[...], hook_context={...})`) and read by the hook with
`state.data.get("hook_context")`.
This hook only makes sense at the `before_tool` hook point, where the pending tool calls exist (between the model
requesting tools and those tools running); the Agent enforces this and raises if it is registered elsewhere. Use a
single ConfirmationHook with one entry per tool (or per tuple of tools) in `confirmation_strategies` rather than
registering several hooks.
#### __init__
```python
__init__(
confirmation_strategies: dict[str | tuple[str, ...], ConfirmationStrategy],
) -> None
```
Initialize the hook with its per-tool confirmation strategies.
**Parameters:**
- **confirmation_strategies** (<code>dict\[str | tuple\[str, ...\], ConfirmationStrategy\]</code>) – Mapping of tool name (or a tuple of tool names) to its `ConfirmationStrategy`.
The wildcard key `"*"` applies to any tool without a more specific entry.
#### run
```python
run(state: State) -> None
```
Confirm the pending tool calls, rewriting the `messages` in `state` to reflect modifications and rejections.
**Parameters:**
- **state** (<code>State</code>) – The Agent's live `State`. Reads the available tools (`state.data.get("tools")`) and the per-run
context (`state.data.get("hook_context")`), and the pending tool calls from the last message; writes the
updated conversation back to `messages`. Reads go through `state.data` rather than `state.get`, which
deep-copies and would break non-copyable resources (e.g. a WebSocket or client) in `hook_context`.
#### run_async
```python
run_async(state: State) -> None
```
Async version of `run`.
#### to_dict
```python
to_dict() -> dict[str, Any]
```
Serialize the hook, including its confirmation strategies (tuple keys become JSON-array strings).
#### from_dict
```python
from_dict(data: dict[str, Any]) -> ConfirmationHook
```
Deserialize the hook, reconstructing its confirmation strategies.
## policies
### AlwaysAskPolicy
Bases: <code>ConfirmationPolicy</code>
Always ask for confirmation.
#### should_ask
```python
should_ask(
tool_name: str, tool_description: str, tool_params: dict[str, Any]
) -> bool
```
Always ask for confirmation before executing the tool.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
**Returns:**
- <code>bool</code> – Always returns True, indicating confirmation is needed.
### NeverAskPolicy
Bases: <code>ConfirmationPolicy</code>
Never ask for confirmation.
#### should_ask
```python
should_ask(
tool_name: str, tool_description: str, tool_params: dict[str, Any]
) -> bool
```
Never ask for confirmation, always proceed with tool execution.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
**Returns:**
- <code>bool</code> – Always returns False, indicating no confirmation is needed.
### AskOncePolicy
Bases: <code>ConfirmationPolicy</code>
Ask only once per tool with specific parameters.
#### __init__
```python
__init__() -> None
```
Creates an instance of AskOncePolicy.
#### should_ask
```python
should_ask(
tool_name: str, tool_description: str, tool_params: dict[str, Any]
) -> bool
```
Ask for confirmation only once per tool with specific parameters.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
**Returns:**
- <code>bool</code> – True if confirmation is needed, False if already asked with the same parameters.
#### update_after_confirmation
```python
update_after_confirmation(
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
confirmation_result: ConfirmationUIResult,
) -> None
```
Store the tool and parameters if the action was "confirm" to avoid asking again.
This method updates the internal state to remember that the user has already confirmed the execution of the
tool with the given parameters.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool that was executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters that were passed to the tool.
- **confirmation_result** (<code>ConfirmationUIResult</code>) – The result from the confirmation UI.
## strategies
### BlockingConfirmationStrategy
Confirmation strategy that blocks execution to gather user feedback.
#### __init__
```python
__init__(
*,
confirmation_policy: ConfirmationPolicy,
confirmation_ui: ConfirmationUI,
reject_template: str = REJECTION_FEEDBACK_TEMPLATE,
modify_template: str = MODIFICATION_FEEDBACK_TEMPLATE,
user_feedback_template: str = USER_FEEDBACK_TEMPLATE
) -> None
```
Initialize the BlockingConfirmationStrategy with a confirmation policy and UI.
**Parameters:**
- **confirmation_policy** (<code>ConfirmationPolicy</code>) – The confirmation policy to determine when to ask for user confirmation.
- **confirmation_ui** (<code>ConfirmationUI</code>) – The user interface to interact with the user for confirmation.
- **reject_template** (<code>str</code>) – Template for rejection feedback messages. It should include a `{tool_name}` placeholder.
- **modify_template** (<code>str</code>) – Template for modification feedback messages. It should include `{tool_name}` and `{final_tool_params}`
placeholders.
- **user_feedback_template** (<code>str</code>) – Template for user feedback messages. It should include a `{feedback}` placeholder.
#### run
```python
run(
*,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
tool_call_id: str | None = None,
confirmation_strategy_context: dict[str, Any] | None = None
) -> ToolExecutionDecision
```
Run the human-in-the-loop strategy for a given tool and its parameters.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
- **tool_call_id** (<code>str | None</code>) – Optional unique identifier for the tool call. This can be used to track and correlate the decision with a
specific tool invocation.
- **confirmation_strategy_context** (<code>dict\[str, Any\] | None</code>) – Optional dictionary for passing request-scoped resources. Useful in web/server environments
to provide per-request objects (e.g., WebSocket connections, async queues, Redis pub/sub clients)
that strategies can use for non-blocking user interaction.
**Returns:**
- <code>ToolExecutionDecision</code> – A ToolExecutionDecision indicating whether to execute the tool with the given parameters, or a
feedback message if rejected.
#### run_async
```python
run_async(
*,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
tool_call_id: str | None = None,
confirmation_strategy_context: dict[str, Any] | None = None
) -> ToolExecutionDecision
```
Async version of run. Calls the sync run() method by default.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
- **tool_call_id** (<code>str | None</code>) – Optional unique identifier for the tool call.
- **confirmation_strategy_context** (<code>dict\[str, Any\] | None</code>) – Optional dictionary for passing request-scoped resources.
**Returns:**
- <code>ToolExecutionDecision</code> – A ToolExecutionDecision indicating whether to execute the tool with the given parameters.
#### to_dict
```python
to_dict() -> dict[str, Any]
```
Serializes the BlockingConfirmationStrategy to a dictionary.
**Returns:**
- <code>dict\[str, Any\]</code> – Dictionary with serialized data.
#### from_dict
```python
from_dict(data: dict[str, Any]) -> BlockingConfirmationStrategy
```
Deserializes the BlockingConfirmationStrategy from a dictionary.
**Parameters:**
- **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from.
**Returns:**
- <code>BlockingConfirmationStrategy</code> – Deserialized BlockingConfirmationStrategy.
## user_interfaces
### RichConsoleUI
Bases: <code>ConfirmationUI</code>
Rich console interface for user interaction.
#### __init__
```python
__init__(console: Console | None = None) -> None
```
Creates an instance of RichConsoleUI.
#### get_user_confirmation
```python
get_user_confirmation(
tool_name: str, tool_description: str, tool_params: dict[str, Any]
) -> ConfirmationUIResult
```
Get user confirmation for tool execution via rich console prompts.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
**Returns:**
- <code>ConfirmationUIResult</code> – ConfirmationUIResult based on user input.
#### to_dict
```python
to_dict() -> dict[str, Any]
```
Serializes the RichConsoleConfirmationUI to a dictionary.
**Returns:**
- <code>dict\[str, Any\]</code> – Dictionary with serialized data.
### SimpleConsoleUI
Bases: <code>ConfirmationUI</code>
Simple console interface using standard input/output.
#### get_user_confirmation
```python
get_user_confirmation(
tool_name: str, tool_description: str, tool_params: dict[str, Any]
) -> ConfirmationUIResult
```
Get user confirmation for tool execution via simple console prompts.
**Parameters:**
- **tool_name** (<code>str</code>) – The name of the tool to be executed.
- **tool_description** (<code>str</code>) – The description of the tool.
- **tool_params** (<code>dict\[str, Any\]</code>) – The parameters to be passed to the tool.
docs-website/sidebars.js
+12 -3
View File
@@ -156,9 +156,18 @@ export default {
label: 'Agents',
items: [
'pipeline-components/agents-1/agent',
'pipeline-components/agents-1/hooks',
'pipeline-components/agents-1/human-in-the-loop',
'pipeline-components/agents-1/tool-result-offloading',
{
type: 'category',
label: 'Hooks',
link: {
type: 'doc',
id: 'pipeline-components/agents-1/hooks',
},
items: [
'pipeline-components/agents-1/human-in-the-loop',
'pipeline-components/agents-1/tool-result-offloading',
],
},
'pipeline-components/agents-1/state',
],
},
haystack/human_in_the_loop/__init__.py → haystack/hooks/human_in_the_loop/__init__.py
View File
haystack/human_in_the_loop/dataclasses.py → haystack/hooks/human_in_the_loop/dataclasses.py
View File
haystack/human_in_the_loop/hooks.py → haystack/hooks/human_in_the_loop/hooks.py
+3 -3
View File
@@ -7,13 +7,13 @@ from typing import Any
from haystack.components.agents.state.state import State
from haystack.components.agents.state.state_utils import replace_values
from haystack.core.serialization import default_from_dict, default_to_dict
from haystack.human_in_the_loop.strategies import (
from haystack.hooks.human_in_the_loop.strategies import (
_deserialize_confirmation_strategies,
_process_confirmation_strategies,
_process_confirmation_strategies_async,
_serialize_confirmation_strategies,
)
from haystack.human_in_the_loop.types import ConfirmationStrategy
from haystack.hooks.human_in_the_loop.types import ConfirmationStrategy
class ConfirmationHook:
@@ -24,7 +24,7 @@ class ConfirmationHook:
```python
from haystack.components.agents import Agent
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
haystack/human_in_the_loop/policies.py → haystack/hooks/human_in_the_loop/policies.py
+2 -2
View File
@@ -4,8 +4,8 @@
from typing import Any
from haystack.human_in_the_loop import ConfirmationUIResult
from haystack.human_in_the_loop.types import ConfirmationPolicy
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from haystack.hooks.human_in_the_loop.types import ConfirmationPolicy
class AlwaysAskPolicy(ConfirmationPolicy):
haystack/human_in_the_loop/strategies.py → haystack/hooks/human_in_the_loop/strategies.py
+2 -2
View File
@@ -9,8 +9,8 @@ from typing import Any
from haystack.components.agents.state.state import State
from haystack.core.serialization import component_to_dict, default_from_dict, default_to_dict
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.human_in_the_loop import ToolExecutionDecision
from haystack.human_in_the_loop.types import ConfirmationPolicy, ConfirmationStrategy, ConfirmationUI
from haystack.hooks.human_in_the_loop import ToolExecutionDecision
from haystack.hooks.human_in_the_loop.types import ConfirmationPolicy, ConfirmationStrategy, ConfirmationUI
from haystack.tools import Tool
from haystack.utils.deserialization import deserialize_component_inplace
haystack/human_in_the_loop/types/__init__.py → haystack/hooks/human_in_the_loop/types/__init__.py
View File
haystack/human_in_the_loop/types/protocol.py → haystack/hooks/human_in_the_loop/types/protocol.py
+1 -1
View File
@@ -5,7 +5,7 @@
from typing import Any, Protocol
from haystack.core.serialization import default_from_dict, default_to_dict
from haystack.human_in_the_loop.dataclasses import ConfirmationUIResult, ToolExecutionDecision
from haystack.hooks.human_in_the_loop.dataclasses import ConfirmationUIResult, ToolExecutionDecision
class ConfirmationUI(Protocol):
haystack/human_in_the_loop/user_interfaces.py → haystack/hooks/human_in_the_loop/user_interfaces.py
+2 -2
View File
@@ -7,8 +7,8 @@ from threading import Lock
from typing import Any
from haystack.core.serialization import default_to_dict
from haystack.human_in_the_loop import ConfirmationUIResult
from haystack.human_in_the_loop.types import ConfirmationUI
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from haystack.hooks.human_in_the_loop.types import ConfirmationUI
from haystack.lazy_imports import LazyImport
with LazyImport(message="Run 'pip install rich'") as rich_import:
pydoc/hooks_api.yml
+5 -3
View File
@@ -1,7 +1,9 @@
loaders:
- search_path: [../haystack/hooks]
modules: ["protocol", "from_function", "tool_result_offloading/hooks", "tool_result_offloading/policies",
"tool_result_offloading/stores", "tool_result_offloading/types/protocol"]
modules: ["protocol", "from_function", "human_in_the_loop/dataclasses", "human_in_the_loop/hooks",
"human_in_the_loop/policies", "human_in_the_loop/strategies", "human_in_the_loop/user_interfaces",
"tool_result_offloading/hooks", "tool_result_offloading/policies", "tool_result_offloading/stores",
"tool_result_offloading/types/protocol"]
processors:
- type: filter
documented_only: true
@@ -10,5 +12,5 @@ renderer:
title: Hooks
id: hooks-api
description: Hooks that run at points in the Agent's run loop and influence it by mutating State, including
built-in tool result offloading.
built-in tool result offloading and Human-in-the-Loop tool confirmation.
filename: hooks_api.md
pydoc/human_in_the_loop_api.yml
-13
View File
@@ -1,13 +0,0 @@
loaders:
- search_path: [../haystack/human_in_the_loop]
modules: ["dataclasses", "hooks", "policies", "strategies", "user_interfaces"]
processors:
- type: filter
documented_only: true
skip_empty_modules: true
renderer:
title: Human-in-the-Loop
id: human-in-the-loop-api
description: Abstractions for integrating human feedback and interaction into Agent
workflows.
filename: human_in_the_loop_api.md
pyproject.toml
+1 -1
View File
@@ -149,7 +149,7 @@ all = 'pytest {args:test}'
e2e = 'pytest {args:e2e}'
# TODO We want to eventually type the whole test folder
types = "mypy --install-types --non-interactive --cache-dir=.mypy_cache/ {args:haystack test/core/ test/marshal/ test/testing/ test/tracing/ test/tools/ test/human_in_the_loop test/evaluation test/document_stores test/dataclasses test/utils/ test/components/embedders/}"
types = "mypy --install-types --non-interactive --cache-dir=.mypy_cache/ {args:haystack test/core/ test/marshal/ test/testing/ test/tracing/ test/tools/ test/hooks/human_in_the_loop test/evaluation test/document_stores test/dataclasses test/utils/ test/components/embedders/}"
[project.urls]
"CI: GitHub" = "https://github.com/deepset-ai/haystack/actions"
releasenotes/notes/recast-hitl-as-before-tool-hook-0ef6ba1502cf97ad.yaml
+4 -2
View File
@@ -6,12 +6,14 @@ upgrade:
removed from ``Agent.__init__``, ``Agent.run``, and ``Agent.run_async``. To migrate, wrap your confirmation
strategies in the new ``ConfirmationHook``, register it under the ``before_tool`` hook point, and pass any
request-scoped resources through the generic ``hook_context`` run argument instead of
``confirmation_strategy_context``:
``confirmation_strategy_context``. The Human-in-the-Loop module has also moved from
``haystack.human_in_the_loop`` to ``haystack.hooks.human_in_the_loop``, so that it lives alongside the other
built-in hooks (such as tool result offloading); update your imports to the new location:
.. code-block:: python
from haystack.components.agents import Agent
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
test/components/agents/test_agent_hitl.py
+8 -6
View File
@@ -12,7 +12,7 @@ from haystack import component
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
@@ -20,7 +20,7 @@ from haystack.human_in_the_loop import (
NeverAskPolicy,
SimpleConsoleUI,
)
from haystack.human_in_the_loop.types import ConfirmationStrategy, ConfirmationUI
from haystack.hooks.human_in_the_loop.types import ConfirmationStrategy, ConfirmationUI
from haystack.tools import Tool, Toolset, create_tool_from_function
@@ -125,18 +125,20 @@ class TestAgent:
"hooks": {
"before_tool": [
{
"type": "haystack.human_in_the_loop.hooks.ConfirmationHook",
"type": "haystack.hooks.human_in_the_loop.hooks.ConfirmationHook",
"init_parameters": {
"confirmation_strategies": {
"addition_tool": {
"type": "haystack.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"type": "haystack.hooks.human_in_the_loop.strategies."
"BlockingConfirmationStrategy",
"init_parameters": {
"confirmation_policy": {
"type": "haystack.human_in_the_loop.policies.NeverAskPolicy",
"type": "haystack.hooks.human_in_the_loop.policies.NeverAskPolicy",
"init_parameters": {},
},
"confirmation_ui": {
"type": "haystack.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"type": "haystack.hooks.human_in_the_loop.user_interfaces."
"SimpleConsoleUI",
"init_parameters": {},
},
"reject_template": "Tool execution for '{tool_name}' was rejected by "
test/hooks/human_in_the_loop/__init__.py
+3
View File
@@ -0,0 +1,3 @@
# SPDX-FileCopyrightText: 2022-present deepset GmbH <[email protected]>
#
# SPDX-License-Identifier: Apache-2.0
test/human_in_the_loop/test_dataclasses.py → test/hooks/human_in_the_loop/test_dataclasses.py
+1 -1
View File
@@ -2,7 +2,7 @@
#
# SPDX-License-Identifier: Apache-2.0
from haystack.human_in_the_loop import ConfirmationUIResult, ToolExecutionDecision
from haystack.hooks.human_in_the_loop import ConfirmationUIResult, ToolExecutionDecision
class TestConfirmationUIResult:
test/human_in_the_loop/test_hooks.py → test/hooks/human_in_the_loop/test_hooks.py
+6 -6
View File
@@ -10,7 +10,7 @@ import pytest
from haystack.components.agents.state.state import State
from haystack.components.agents.state.state_utils import merge_lists, replace_values
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
@@ -19,7 +19,7 @@ from haystack.human_in_the_loop import (
SimpleConsoleUI,
ToolExecutionDecision,
)
from haystack.human_in_the_loop.types import ConfirmationUI
from haystack.hooks.human_in_the_loop.types import ConfirmationUI
from haystack.tools import Tool, create_tool_from_function
@@ -159,18 +159,18 @@ class TestConfirmationHook:
}
)
assert hook.to_dict() == {
"type": "haystack.human_in_the_loop.hooks.ConfirmationHook",
"type": "haystack.hooks.human_in_the_loop.hooks.ConfirmationHook",
"init_parameters": {
"confirmation_strategies": {
"addition_tool": {
"type": "haystack.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"type": "haystack.hooks.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"init_parameters": {
"confirmation_policy": {
"type": "haystack.human_in_the_loop.policies.NeverAskPolicy",
"type": "haystack.hooks.human_in_the_loop.policies.NeverAskPolicy",
"init_parameters": {},
},
"confirmation_ui": {
"type": "haystack.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"type": "haystack.hooks.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"init_parameters": {},
},
"reject_template": "Tool execution for '{tool_name}' was rejected by the user.",
test/human_in_the_loop/test_policies.py → test/hooks/human_in_the_loop/test_policies.py
+7 -7
View File
@@ -4,7 +4,7 @@
import pytest
from haystack.human_in_the_loop import AlwaysAskPolicy, AskOncePolicy, ConfirmationUIResult, NeverAskPolicy
from haystack.hooks.human_in_the_loop import AlwaysAskPolicy, AskOncePolicy, ConfirmationUIResult, NeverAskPolicy
from haystack.tools import Tool, create_tool_from_function
@@ -25,11 +25,11 @@ class TestAlwaysAskPolicy:
def test_to_dict(self):
policy = AlwaysAskPolicy()
policy_dict = policy.to_dict()
assert policy_dict["type"] == "haystack.human_in_the_loop.policies.AlwaysAskPolicy"
assert policy_dict["type"] == "haystack.hooks.human_in_the_loop.policies.AlwaysAskPolicy"
assert policy_dict["init_parameters"] == {}
def test_from_dict(self):
policy_dict = {"type": "haystack.human_in_the_loop.policies.AlwaysAskPolicy", "init_parameters": {}}
policy_dict = {"type": "haystack.hooks.human_in_the_loop.policies.AlwaysAskPolicy", "init_parameters": {}}
policy = AlwaysAskPolicy.from_dict(policy_dict)
assert isinstance(policy, AlwaysAskPolicy)
@@ -66,11 +66,11 @@ class TestAskOncePolicy:
def test_to_dict(self):
policy = AskOncePolicy()
policy_dict = policy.to_dict()
assert policy_dict["type"] == "haystack.human_in_the_loop.policies.AskOncePolicy"
assert policy_dict["type"] == "haystack.hooks.human_in_the_loop.policies.AskOncePolicy"
assert policy_dict["init_parameters"] == {}
def test_from_dict(self):
policy_dict = {"type": "haystack.human_in_the_loop.policies.AskOncePolicy", "init_parameters": {}}
policy_dict = {"type": "haystack.hooks.human_in_the_loop.policies.AskOncePolicy", "init_parameters": {}}
policy = AskOncePolicy.from_dict(policy_dict)
assert isinstance(policy, AskOncePolicy)
@@ -83,10 +83,10 @@ class TestNeverAskPolicy:
def test_to_dict(self):
policy = NeverAskPolicy()
policy_dict = policy.to_dict()
assert policy_dict["type"] == "haystack.human_in_the_loop.policies.NeverAskPolicy"
assert policy_dict["type"] == "haystack.hooks.human_in_the_loop.policies.NeverAskPolicy"
assert policy_dict["init_parameters"] == {}
def test_from_dict(self):
policy_dict = {"type": "haystack.human_in_the_loop.policies.NeverAskPolicy", "init_parameters": {}}
policy_dict = {"type": "haystack.hooks.human_in_the_loop.policies.NeverAskPolicy", "init_parameters": {}}
policy = NeverAskPolicy.from_dict(policy_dict)
assert isinstance(policy, NeverAskPolicy)
test/human_in_the_loop/test_strategies.py → test/hooks/human_in_the_loop/test_strategies.py
+8 -8
View File
@@ -10,7 +10,7 @@ import pytest
from haystack.components.agents.state.state import State
from haystack.components.agents.tool_calling import ToolNotFoundException, _run_tool
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.human_in_the_loop import (
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
AskOncePolicy,
BlockingConfirmationStrategy,
@@ -19,7 +19,7 @@ from haystack.human_in_the_loop import (
SimpleConsoleUI,
ToolExecutionDecision,
)
from haystack.human_in_the_loop.strategies import (
from haystack.hooks.human_in_the_loop.strategies import (
_apply_tool_execution_decisions,
_process_confirmation_strategies,
_run_confirmation_strategies,
@@ -60,14 +60,14 @@ class TestBlockingConfirmationStrategy:
strategy = BlockingConfirmationStrategy(confirmation_policy=AskOncePolicy(), confirmation_ui=SimpleConsoleUI())
strategy_dict = strategy.to_dict()
assert strategy_dict == {
"type": "haystack.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"type": "haystack.hooks.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"init_parameters": {
"confirmation_policy": {
"type": "haystack.human_in_the_loop.policies.AskOncePolicy",
"type": "haystack.hooks.human_in_the_loop.policies.AskOncePolicy",
"init_parameters": {},
},
"confirmation_ui": {
"type": "haystack.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"type": "haystack.hooks.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"init_parameters": {},
},
"reject_template": "Tool execution for '{tool_name}' was rejected by the user.",
@@ -79,14 +79,14 @@ class TestBlockingConfirmationStrategy:
def test_from_dict(self):
strategy_dict = {
"type": "haystack.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"type": "haystack.hooks.human_in_the_loop.strategies.BlockingConfirmationStrategy",
"init_parameters": {
"confirmation_policy": {
"type": "haystack.human_in_the_loop.policies.AskOncePolicy",
"type": "haystack.hooks.human_in_the_loop.policies.AskOncePolicy",
"init_parameters": {},
},
"confirmation_ui": {
"type": "haystack.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"type": "haystack.hooks.human_in_the_loop.user_interfaces.SimpleConsoleUI",
"init_parameters": {},
},
},
test/human_in_the_loop/test_user_interfaces.py → test/hooks/human_in_the_loop/test_user_interfaces.py
+10 -8
View File
@@ -6,7 +6,7 @@ from unittest.mock import MagicMock, patch
import pytest
from haystack.human_in_the_loop import ConfirmationUIResult, RichConsoleUI, SimpleConsoleUI
from haystack.hooks.human_in_the_loop import ConfirmationUIResult, RichConsoleUI, SimpleConsoleUI
from haystack.tools import create_tool_from_function
@@ -26,7 +26,7 @@ class TestRichConsoleUI:
def test_process_choice_confirm(self, tool, choice):
ui = RichConsoleUI(console=MagicMock())
with patch("haystack.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=[choice, "feedback"]):
with patch("haystack.hooks.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=[choice, "feedback"]):
result = ui.get_user_confirmation(tool.name, tool.description, {"x": 1})
assert isinstance(result, ConfirmationUIResult)
@@ -38,7 +38,7 @@ class TestRichConsoleUI:
def test_process_choice_modify(self, tool, choice):
ui = RichConsoleUI(console=MagicMock())
with patch("haystack.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["m", "2"]):
with patch("haystack.hooks.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["m", "2"]):
result = ui.get_user_confirmation(tool.name, tool.description, {"x": 1})
assert isinstance(result, ConfirmationUIResult)
@@ -48,7 +48,9 @@ class TestRichConsoleUI:
def test_process_choice_modify_dict_param(self, tool):
ui = RichConsoleUI(console=MagicMock())
with patch("haystack.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["m", '{"key": "value"}']):
with patch(
"haystack.hooks.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["m", '{"key": "value"}']
):
result = ui.get_user_confirmation(tool.name, tool.description, {"param1": {"old_key": "old_value"}})
assert isinstance(result, ConfirmationUIResult)
@@ -59,7 +61,7 @@ class TestRichConsoleUI:
ui = RichConsoleUI(console=MagicMock())
with patch(
"haystack.human_in_the_loop.user_interfaces.Prompt.ask",
"haystack.hooks.human_in_the_loop.user_interfaces.Prompt.ask",
side_effect=["m", "invalid_json", '{"key": "value"}'],
):
result = ui.get_user_confirmation(tool.name, tool.description, {"param1": {"old_key": "old_value"}})
@@ -72,7 +74,7 @@ class TestRichConsoleUI:
def test_process_choice_reject(self, tool, choice):
ui = RichConsoleUI(console=MagicMock())
with patch("haystack.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["n", "Changed my mind"]):
with patch("haystack.hooks.human_in_the_loop.user_interfaces.Prompt.ask", side_effect=["n", "Changed my mind"]):
result = ui.get_user_confirmation(tool.name, tool.description, {"x": 1})
assert isinstance(result, ConfirmationUIResult)
@@ -82,7 +84,7 @@ class TestRichConsoleUI:
def test_to_dict(self):
ui = RichConsoleUI()
data = ui.to_dict()
assert data["type"] == ("haystack.human_in_the_loop.user_interfaces.RichConsoleUI")
assert data["type"] == ("haystack.hooks.human_in_the_loop.user_interfaces.RichConsoleUI")
assert data["init_parameters"]["console"] is None
def test_from_dict(self):
@@ -181,7 +183,7 @@ class TestSimpleConsoleUI:
def test_to_dict(self):
ui = SimpleConsoleUI()
data = ui.to_dict()
assert data["type"] == ("haystack.human_in_the_loop.user_interfaces.SimpleConsoleUI")
assert data["type"] == ("haystack.hooks.human_in_the_loop.user_interfaces.SimpleConsoleUI")
assert data["init_parameters"] == {}
def test_from_dict(self):