Add DocumentTransformer abstraction so that in #2915 we don't have to
wrap TextSplitter and RedundantEmbeddingFilter (neither of which uses
the query) in the contextual doc compression abstractions. with this
change, doc filter (doc extractor, whatever we call it) would look
something like
```python
class BaseDocumentFilter(BaseDocumentTransformer[_RetrievedDocument], ABC):
@abstractmethod
def filter(self, documents: List[_RetrievedDocument], query: str) -> List[_RetrievedDocument]:
...
def transform_documents(self, documents: List[_RetrievedDocument], query: Optional[str] = None, **kwargs: Any) -> List[_RetrievedDocument]:
if query is None:
raise ValueError("Must pass in non-null query to DocumentFilter")
return self.filter(documents, query)
```
Currently, the output type of a number of OutputParser's `parse` methods
is `Any` when it can in fact be inferred.
This PR makes BaseOutputParser use a generic type and fixes the output
types of the following parsers:
- `PydanticOutputParser`
- `OutputFixingParser`
- `RetryOutputParser`
- `RetryWithErrorOutputParser`
The output of the `StructuredOutputParser` is corrected from `BaseModel`
to `Any` since there are no type guarantees provided by the parser.
Fixes issue #2715
@3coins + @zoltan-fedor.... heres the pr + some minor changes i made.
thoguhts? can try to get it into tmrws release
---------
Co-authored-by: Zoltan Fedor <zoltan.0.fedor@gmail.com>
Co-authored-by: Piyush Jain <piyushjain@duck.com>
* add implementations of `BaseCallbackHandler` to support tracing:
`SharedTracer` which is thread-safe and `Tracer` which is not and is
meant to be used locally.
* Tracers persist runs to locally running `langchain-server`
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Add `finish_reason` to `Generation` as well as extend
`BaseOpenAI._generate` to include it in the output. This can be useful
for usage in downstream tasks when we need to filter for only
generations that finished because of `"stop"` for example. Maybe we
should add this to `LLMChain` as well?
For more details, see
https://beta.openai.com/docs/guides/completion/best-practices
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Davis Chase