Differentiate real human input from code output #2

New Issue

2026-02-16T07:16:28-05:00

yindo commented

2026-02-16 07:16:28 -05:00

Originally created by @SuperMuel on GitHub (Mar 27, 2025).

Currently, the result of the code execution is shown as an HumamMessage to the LLM with no context. The LLM (especially small LLMs like 4o-mini in this case) might confuse a tool output with a real human message.

The following example illustrates this.

I defined a simple tool print_secret which prints a secret to the console. I expect the LLM to execute that tool, read the output and tell me the secret in the last AI message.

When the output of the code is just given as an HumanMessage, the LLM answer with Thank you for sharing the secret message! If you have any further questions or tasks, feel free to ask.

https://smith.langchain.com/public/2e4090a0-4099-4bd6-bb2c-d42a17d27340/r

But when I wrap the code output with <output> tags and with an additional indication, I get the expected output The secret message is...

https://smith.langchain.com/public/4d4b5173-66f1-4fa4-b0fe-d88adc62a11f/r

Thus, I suggest we add an indication that explain to the LLM that we are showing the code output, not a query from the user. In my example, I did :

    def sandbox(state: CodeActState):
       # ...

        return {
            "messages": [
                {
                    "role": "user",
                    "content": f"Here's the output of the script:\n\n<output>\n{output}\n</output>", # <- here 
                }
            ],
            "context": new_context,
        }

Originally created by @SuperMuel on GitHub (Mar 27, 2025). Currently, the result of the code execution is shown as an HumamMessage to the LLM with no context. The LLM (especially small LLMs like `4o-mini` in this case) might confuse a tool output with a real human message. The following example illustrates this. I defined a simple tool `print_secret` which prints a secret to the console. I expect the LLM to execute that tool, read the output and tell me the secret in the last AI message. When the output of the code is just given as an HumanMessage, the LLM answer with `Thank you for sharing the secret message! If you have any further questions or tasks, feel free to ask.` https://smith.langchain.com/public/2e4090a0-4099-4bd6-bb2c-d42a17d27340/r But when I wrap the code output with `<output>` tags and with an additional indication, I get the expected output `The secret message is...` https://smith.langchain.com/public/4d4b5173-66f1-4fa4-b0fe-d88adc62a11f/r --- Thus, I suggest we add an indication that explain to the LLM that we are showing the code output, not a query from the user. In my example, I did : ```python def sandbox(state: CodeActState): # ... return { "messages": [ { "role": "user", "content": f"Here's the output of the script:\n\n<output>\n{output}\n</output>", # <- here } ], "context": new_context, } ```

yindo commented

2026-02-16 07:16:28 -05:00