- Call info within tracing queue
- Default to batching even if info endpoint fails
If you turn batching off, you have to make your own tracer / traceable
flow that handles threading yourself now.
This lets you scope hide inputs/outputs to a client. It also lets you
add custom filters (rather than just masking everything.
You could then use this in the traceable decorator or langchain tracer
to filter runs within a given trace.
It would _not_ facilitate filtering on a span level though, since it
only sees the inputs/outputs and the client is shared within a trace.
Alternatives could include a single function (filter_run) that takes in
a Run object, though then we'd have to re-construct the run within the
client, which is a bit annoying
So you get linter warnings if you try to do a name. It doesn't actually
alter the behavior, so if we choose to extend the number of types, it
would just be a linting issue rather than a runtime issue
Change to let you do this:
```
from langsmith import traceable
@traceable
def foo():
return "bar"
```
Rather than ONLY this:
```
from langsmith import traceable
@traceable() # <- need to call the function
def foo():
return "bar"
```
We plan to introduce an environment variable in hosted langserve that
will overwrite the project used for traces from hosted deployments. We
don't centralize how we refer to projects in any of our repos, so in
order to ensure that our tracing works properly and our project
selection is consistent, we need to introduce a shared method for
getting the project that we can use throughout our different repos
For generator functions, log a list of outputs by default. Let user
define a reduce_fn within the decorated function to support custom
behavior (e.g., if you wanted to reduce OAI dictionaries and only log
the resulting string.
Context: feedback is entering the write queue and so we can't get the
info back in the response.
- Update the return object to be populated from the creation object
rather than the API response.
- Permit manual specification of the ID's
- Update the evaluate_run() return object to be the evaluationresult
William FH