langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

[PR #5988] test(prebuilt): standard integration tests for create_agent #4768

New Issue
Closed
opened 2026-02-20 17:50:43 -05:00 by yindo · 0 comments
yindo commented 2026-02-20 17:50:43 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langchain-ai/langgraph/pull/5988

State: closed
Merged: Yes

This PR improves standard integration tests in prebuilt to ensure logical equivalence between the Python and JavaScript implementations of create_agent.

  • Cleans up test_responses_int test harness
  • Adds new test utils to dynamically load JSON test specs
  • Adds test_return_direct_int test harness to validate model behavior when return_direct tool property is set
  • Adds support for when the user instantiates ToolOutput with multiple JSON schemas unified by the oneOf keyword

Failing tests:

  • test_inference_to_native_output: there is some odd behavior where the model makes a second call to get_weather despite having just received the tool message, so there are 6 messages instead of the 4 expected.
  • test_responses_integration_matrix[asking for information that does not fit into the response format]: XFAIL, currently failing due to undefined behavior when the model cannot conform to any of the structured response formats.

TODO in future PRs:

  • Add exception handling to pass test_responses_integration_matrix.
**Original Pull Request:** https://github.com/langchain-ai/langgraph/pull/5988 **State:** closed **Merged:** Yes --- This PR improves standard integration tests in prebuilt to ensure logical equivalence between the Python and JavaScript implementations of `create_agent`. * Cleans up `test_responses_int` test harness * Adds new test utils to dynamically load JSON test specs * Adds `test_return_direct_int` test harness to validate model behavior when `return_direct` tool property is set * Adds support for when the user instantiates `ToolOutput` with multiple JSON schemas unified by the `oneOf` keyword Failing tests: * `test_inference_to_native_output`: there is some odd behavior where the model makes a second call to `get_weather` despite having just received the tool message, so there are 6 messages instead of the 4 expected. * `test_responses_integration_matrix[asking for information that does not fit into the response format]`: `XFAIL`, currently failing due to undefined behavior when the model cannot conform to any of the structured response formats. TODO in future PRs: * Add exception handling to pass `test_responses_integration_matrix`.
yindo added the pull-request label 2026-02-20 17:50:43 -05:00
yindo closed this issue 2026-02-20 17:50:43 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#4768
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?