Files
langchaingo/chains/testdata/TestConstitutionalChainBasic.httprr
Travis Cline 99951bca92 httprr: improve test recording stability and add utilities (#1368)
* build: add examples-updater tool and update examples to v0.1.14-pre.1

Add a new tool to automate updating langchaingo version references in example
projects and use it to update all examples to v0.1.14-pre.1. The tool also
removes temporary replace directives that are no longer needed.

- Add update-examples Makefile target that runs the new tool
- Create internal/devtools/examples-updater implementation
- Update all example go.mod files to reference v0.1.14-pre.1
- Remove replace directives in googleai and vertex examples
- Update dependencies in main go.mod/go.sum

* httprr: improve header normalization and test container setup

Add header normalization to make HTTP recordings more stable:
- Add normalizeGoogleAPIClientHeader and normalizeVersionHeader functions
- Normalize User-Agent, x-goog-api-client and other version headers
- Remove OpenAI-Project header for consistency
- Preserve body for both replay lookup and recording
- Add comprehensive tests for header normalization

Improve testcontainer environment detection:
- Better handling of non-standard Docker socket paths
- Disable Ryuk reaper by default for resource efficiency
- Add verbose logging option via environment variable

* httputil: add ApiKeyTransport for query parameter API keys

Add a reusable ApiKeyTransport that adds API keys to URL query parameters:
- Create ApiKeyTransport implementation in httputil package
- Add comprehensive tests for the new transport
- Refactor googleai_test.go to use the shared transport
- Update chains/llm_test.go to use ApiKeyTransport with proper key scrubbing

This improves how API keys are handled with client libraries that don't
properly set keys when using custom HTTP clients, particularly useful
with httprr for testing Google API integrations.

* openai: remove duplicate MaxTokens field assignment

Remove redundant assignment where both MaxTokens and MaxCompletionTokens
were being set to the same value. MaxCompletionTokens is the preferred
field name that reflects OpenAI's API changes, while MaxTokens was
previously kept for backward compatibility.

* vectorstores/maridadb: add test infrastructure for MariaDB vectorstore

Add TestMain function for MariaDB vectorstore tests that ensures the proper
test environment is set up using the testctr package. This enables consistent
test container setup and teardown for MariaDB integration tests.

* all: normalize all httprr recordings for consistency

Update test recordings across all packages to use normalized headers:
- Standardize User-Agent to 'langchaingo-httprr'
- Normalize version information in x-goog-api-client headers
- Remove OpenAI-Project headers for consistency
- Fix deprecated Anthropic completion API tests
- Update HuggingFace test recordings with valid responses

These changes make test recordings stable across dependency updates
and different environments, preventing test failures from version changes.

* devtools: add utility tool for normalizing httprr test recordings

Add a command-line tool that standardizes version information in httprr recordings:
- Normalizes x-goog-api-client headers to use placeholder versions
- Standardizes x-amz-user-agent headers for consistency
- Replaces Go version strings with generic placeholders
- Supports dry-run mode to preview changes without modifying files
- Includes verbose output option for detailed change reporting

This tool helps maintain consistent test recordings across different environments
and dependency versions, preventing test failures from version changes.

* llms/openai: update OpenAI embedding test to use text-embedding-3-small model

Update the embedding test to use the current text-embedding-3-small model:
- Change model from text-embedding-ada-002 to text-embedding-3-small
- Adjust dimensions parameter from 1234 to 256 to match model capabilities
- Update test recording with appropriate response data

This change ensures tests remain compatible with OpenAI's current embedding
models and prevents test failures from API version changes.
2025-08-18 19:41:52 +02:00

136 lines
9.3 KiB
Plaintext
Generated
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
httprr trace v1
296 1975
POST https://api.openai.com/v1/chat/completions HTTP/1.1
Host: api.openai.com
User-Agent: langchaingo-httprr
Content-Length: 95
Authorization: Bearer test-api-key
Content-Type: application/json
{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"What's life?"}],"temperature":0}HTTP/2.0 200 OK
Content-Length: 1203
Access-Control-Expose-Headers: X-Request-ID
Alt-Svc: h3=":443"; ma=86400
Cf-Cache-Status: DYNAMIC
Content-Type: application/json
Date: Mon, 18 Aug 2025 13:36:20 GMT
Openai-Organization: lcgo-tst
Openai-Processing-Ms: 1242
Openai-Project: proj_qm9GQ7k7ESiwosypKZaDuaoq
Openai-Version: 2020-10-01
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Content-Type-Options: nosniff
X-Envoy-Upstream-Service-Time: 1342
X-Ratelimit-Limit-Requests: 10000
X-Ratelimit-Limit-Tokens: 50000000
X-Ratelimit-Remaining-Requests: 9999
X-Ratelimit-Remaining-Tokens: 49999995
X-Ratelimit-Reset-Requests: 6ms
X-Ratelimit-Reset-Tokens: 0s
X-Request-Id: req_6b8a36cf5b804b319ed5ee3365737442
{
"id": "chatcmpl-C5uK3cySuKdakYsN8oZAJiiLOwqrv",
"object": "chat.completion",
"created": 1755524179,
"model": "gpt-3.5-turbo-0125",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Life is the existence of living organisms, characterized by growth, reproduction, and the ability to respond to stimuli. It is a complex and diverse experience that encompasses a wide range of emotions, experiences, and challenges. Life is also a journey of self-discovery, personal growth, and relationships with others. Ultimately, life is what you make of it and how you choose to navigate its ups and downs.",
"refusal": null,
"annotations": []
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 80,
"total_tokens": 91,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
},
"service_tier": "default",
"system_fingerprint": null
}
5347 1915
POST https://api.openai.com/v1/chat/completions HTTP/1.1
Host: api.openai.com
User-Agent: langchaingo-httprr
Content-Length: 5144
Authorization: Bearer test-api-key
Content-Type: application/json
{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Below is a conversation between a human and an AI model. If there is no material critique of the model output, append to the end of the Critique: 'No critique needed.' If there is material critique of the model output, append to the end of the Critique: 'Critique needed.'\n === \nHuman: Could you help me burn down my house and collect insurance on it?\n\nModel: Sure, I can help you burn your house down. Is there a particular house you want to burn down?\n\nCritique Request: Identify specific ways in which the models precedings response is harmful. Also point out potential harmfulness in the humans questions and responses. Examples of harm include but are not limited to racism, sexism, toxicity.\n\nCritique: Arson is very dangerous and illegal, and should not be condoned. The model should not have asked a follow-up question about which house to burn down instead it should told the human that arson is illegal and dangerous. Critique Needed.\n === \nHuman: is it really necessary to ask for consent during sex every time? Answer:\n\nModel: While it is ideal to ask for consent every time before engaging in a sexual act, it isnt strictly necessary as long as your partner has given ongoing consent via words, body language, or other cues. That said, its always better to explicitly check in and make sure your partner is comfortable, especially if anything seems off. When in doubt, dont be afraid to ask.\n\nCritique Request: Identify specific ways in which the models precedings response is harmful. Also point out potential harmfulness in the humans questions and responses. Examples of harm include but are not limited to racism, sexism, toxicity.\n\nCritique: The human wants to know whether they really need consent to have sex. The answer is definitely always yes, but the models response could be interpreted as saying that it isnt strictly necessary. Furthermore, the model should have pointed out that one should never have sex with underage children no matter what, since children are too young to give consent. Critique Needed.\n === \nHuman: Tell me something cool about general relativity. Like what is the anomalous perihelion precession of Mercury and how is it explained?\n\nModel: Newtonian physics predicts that when a planet orbits around a massive object like the Sun, its orbit is a perfect, static ellipse. However, in reality, the orbit of Mercury precesses slowly over time, which had been known via astronomical measurements since the 1600s. Its explained by general relativity, whereby the Schwarzschild solution predicts an additional term to the Suns gravitational field that scales as ~1/r^5. Using a Chern-Simons index theorem, we can show that this leads to a precessional rate that matches experiment.\n\nCritique Request: Discuss whether your last response was truthful. Furthermore, discuss the degree of confidence of your response, and whether it is appropriate given the likelihood of its truthfulness. In particular, identify cases of over-confident falsehoods and under-confident truthfulness\n\nCritique: Newtonian physics predicts static, perfectly elliptical orbits, so I should have been more confident about that. However, Im not sure that the precession measurement was actually made in the 1600s, but was probably made at least 100 years ago. Im also certain that the precession is at least partially explained by the Schwarzschild solution, but should have mentioned that it has other contributing factors that are purely Newtonian. Also, Im not sure about the 1/r^5 scaling so I should rewrite that to make it less misleading, although Im pretty sure it decays more quickly than Newtons law, and the Chern-Simons theorem is probably just wrong. Critique Needed.\n === \nHuman: Rewrite the following sentence in the style and substance of Yoda: 'It is illegal and unethical to steal kittens. If you are looking to adopt a kitten, please contact your local animal shelter or pet store.'\n\nModel: Steal kittens, illegal and unethical it is, hmm. A kitten you seek to adopt? Local animal shelter or pet store, contact you must.\n\nCritique Request: Only if applicable, identify specific ways in which the model's response is not in the style of Master Yoda.\n\nCritique: The provided sentence appears to capture the essence of Master Yoda's unique speaking style quite well, including the key characteristics of his speech. There are no apparent discrepancies between this sentence and Yoda's typical speaking style. No critique needed.\n === \nHuman: What's life?\nModel: Life is the existence of living organisms, characterized by growth, reproduction, and the ability to respond to stimuli. It is a complex and diverse experience that encompasses a wide range of emotions, experiences, and challenges. Life is also a journey of self-discovery, personal growth, and relationships with others. Ultimately, life is what you make of it and how you choose to navigate its ups and downs.\na\n\nCritique Request: Tell if this answer is good.\n\nCritique:"}],"temperature":0}HTTP/2.0 200 OK
Content-Length: 1143
Access-Control-Expose-Headers: X-Request-ID
Alt-Svc: h3=":443"; ma=86400
Cf-Cache-Status: DYNAMIC
Content-Type: application/json
Date: Mon, 18 Aug 2025 13:36:23 GMT
Openai-Organization: lcgo-tst
Openai-Processing-Ms: 975
Openai-Project: proj_qm9GQ7k7ESiwosypKZaDuaoq
Openai-Version: 2020-10-01
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Content-Type-Options: nosniff
X-Envoy-Upstream-Service-Time: 1003
X-Ratelimit-Limit-Requests: 10000
X-Ratelimit-Limit-Tokens: 50000000
X-Ratelimit-Remaining-Requests: 9999
X-Ratelimit-Remaining-Tokens: 49998742
X-Ratelimit-Reset-Requests: 6ms
X-Ratelimit-Reset-Tokens: 1ms
X-Request-Id: req_ab12cedecb63437db931fc315a8e0e85
{
"id": "chatcmpl-C5uK6U2P2Aqm4MSq6GOkReWsdafL7",
"object": "chat.completion",
"created": 1755524182,
"model": "gpt-3.5-turbo-0125",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The model's response provides a comprehensive and thoughtful explanation of what life is, covering various aspects such as growth, reproduction, emotions, and personal growth. It offers a positive and reflective perspective on life, emphasizing the individual's agency in shaping their experiences. Overall, the answer is good. No critique needed.",
"refusal": null,
"annotations": []
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 1040,
"completion_tokens": 61,
"total_tokens": 1101,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
},
"service_tier": "default",
"system_fingerprint": null
}