* add faiss
* add pickle support
* replace pickle.js with pickleparser package
* yarn install
* update pickleparser
* fix code review
* fix metadata in loadFromPython
* add docs
* add test cases
* port form python
* Save mapping into JSON file.
* upgrader pickleparser
* upgrade pickleparser
* make loadFromPython work
* upgrade pickleparser
* upgrade pickleparser and use reduce function
* update demo and docs
* fix doc build
* update docs
* add more ut and fix import of pickleparser
* update docs
* add int test for loadFromPython
* add more int test for loadFromPython
* fix merge error
* upgrade pickleparser
* upgrade pickleparser and fix code review
* yarn install
* port #1142
* use chained register
* fix lint
* fix format
* Update examples, update imports, remove extraneous deps from merge
* Undo change to imports
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
* self-query with meriyah
* removed builtin translators so end-user needs to specify their own translator, but a basic translator that can be used with pinecone and chroma is included and can be imported
* fixed class name typo
* moved meriyah to main dependency so expression parser can work in test-exports-*
* added back self_query to entrypoints
* moved meriyah to dev/peer dep, fixed unused parser types, minor fixes
* renaming expression type to reflect meriyah better
* removed unused lint disable on self_query/base.ts
* Use a rewritten tiktoken for token counting
* Fix lint, add test
* Fix type import, typo
* Remove @dqbd/tiktoken as "weak" dependency
* Move JS implementation of tiktoken into a separate package
* Upgrade to js-tiktoken@1.0.2
* Replace all occurrences of @dqbd/tiktoken
* Avoid bundling ranks, fix test with CRA
* Use AsyncCaller, cache promises
* Remove WASM configuration changes
* Upgrade to 1.0.6 for CRA fixes, mark js-tiktoken/lite as an external dependency
* Remove next.config.js
* init entity_memory
* init tests
* added test
* fixed test
* updated prompts usage
* fixed import
* added test for pre loaded messages
* updated import
* Use object without prototype for safer exists check
* move entity store to its own file, make it async
* Add example, not giving great results for now
* Prompt updates to support reflexive pronouns, keep entity summaries unchanged, support chat models, add docs
* Remove merge artifact
* Fix test
* Format fix
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
* feat: add Redis vectorstore with tests
* fix linting and formatting problems
* docs: add redis vector store
* fix: remove export from deprecated vectorstores index
* docs: add node.js-only compatibility tip to redis vectorstore
* Update types, add more optional config, move examples out of md file
* Use more recent chain in example
* Update tests
* Mark entrypoint as optional dep
* Fix import
* Fix build error
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
* Merge post upstream sync (#1)
* Added RedisMemory with tests and docs
* fixed Redis test file name
* TODOs for quick refactor
* updated memory to accept a client
* Added TODOS
* ongoing testing
* Updated tests
* Updated tests for memory return option
* finalized tests + updated docs
* Adding docs
* Readded init and cleaned up func
* fixed test typing
* redo yarn lock
* remove yarn.lock
* updated yarn lock and namespaced redis
* fix merge conflict
* updated BaseChatMemoryInput
* Updated with lint fixes
* Fixed docs to match memory instantiation
* yarn format docs
* Merging fixes to address ForkPR comments (#2)
* updating for pr
* Removed redis_memory in favor of chat memory
* Fixed tests and updated docs
* Bump docs
* lint results
* fixes from lint and format
---------
Co-authored-by: Chris Toomey <ctoomey@fb.com>
* Update RedisChatMessageHistory class to initialize a client internally
* Update Redis chat message history docs to reflect the fact that Redis is a peer dependency, allow direct passing of Redis client
* Patch typing issues with passing node-redis client directly into a RedisChatMessageHistory instance
---------
Co-authored-by: Chris Toomey <ctoomey@fb.com>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
* Implement cancellation and timeout for all LLMs, allow passing LLM call options via LLMChain (like was already possible for stop)
* Update example
* Add declared ParsedCallOptions type for BaseLanguageModels, add signal support for chat models, add tests
* Update docs for chat models
* Update llm and chat examples to remove `run` func
* Add more examples for LLMChain
* Reorganise agent docs, add Additional Functionality doc for Agents
* Add toc in llm chain docs
* Format
---------
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
* Add dynamodb chat message history
* Some updates to deps and interfaces
* Use list_append method to add messages in DynamoDBChatMessageHistory, docs, tests, and other fixups
* Move DynamoDB message store into its own entrypoint, update docs
* Fix typo
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Jacob Lee <jacob@autocode.com>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
* Added filtering ability to supabase
* Supabase metadata filter polish: small fix, adds integration test, update docs
* Skip Supabase integration test reliant on outside setup
* Use FakeEmbeddings in unit test
* Update docs
---------
Co-authored-by: Jacob Lee <jacob@autocode.com>
* feat: support unstructured api key in loader
* add API key to instructions and examples
* change api key order
* update integration tests
* retrigger build
* default web path to hosted api
* moved api key to an options argument
* interface for unstructured options
* linting, linting, linting
* refactor loaders into mapping
* move webpath to options
* update function calls in test
* fix examples
* linting, linting, linting ...
* Adds shim for existing Unstructured users
* Remove bad content type header from Unstructured API call
* Fix md formatting
* Small fixes
* Move additional UnstructuredDirectoryLoader constructor options into an options object
* Remove extra unnecessary declaration
---------
Co-authored-by: Matt Robinson <mrobinson@unstructuredai.io>
Co-authored-by: Matt Robinson <mrobinson@unstructured.io>
Co-authored-by: Jacob Lee <jacob@autocode.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
* added support for Azure OpenAI. Also added other Azure OpenAI parameters to make experience better
* added azure openai to docs and samples
* fixed linting and formatting
* Added tests for exports. merged all different openai input types into a more general types definition
* fixed rebase issues
* cleanup
* Cleanup. Also added two new settings/envvars for azure, so different deployments can be used for chat, completion and embeddings.
* Update embeddings
* Add missing exports
* Do not use types that arent exported
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
* added constitutional chain translated from the main python langchain repo
* added pure for side effect handling
* fixed constitutional chain tests
* forgot formatting last time
* added constitutional chain doc and example
* added moderation chain
* changed the integration test example to something less extreme
* Use fetch adapater, consistent naming, consistent args
* Use async caller
* added example and docs
* formatting fix
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Priya X. Pramesi <ppramesi@visi8.com>
* feat: added returnIntermediateSteps flag to MapReduceChain
* docs: addded extra documentation around the intermediate steps
* fix: linting issue
* docs: set map_reduce in backticks
* feat: added option to loadQAChain as well
* Populate params via destructuring, adds unit test
* Lint
---------
Co-authored-by: Jacob Lee <jacob@autocode.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
* feat: Introduce Time-Weighted Retrieval for Relevant Documents
- Add new file `time_weighted.ts` with functions to retrieve, add, and get documents from a vector store
- Define `TimeWeightedVectorStoreRetriever` class that inherits from `BaseRetriever`
- Include functions to calculate scores and hours passed to help identify salient documents
* test: Add tests for time weighted retrievers.
- Add new test file for time weighted retrievers
- Implement several tests for TimeWeightedVectorStoreRetriever's getRelevantDocuments method with different data and searchKwargs values
* refactor: Refactor TimeWeightedVectorStoreRetriever for better organization.
- Refactored code in TimeWeightedVectorStoreRetriever for better organization
- Combined memoryDocsAndScores and salientDocsAndScores for easy retrieval
- Changed method names for better descriptive naming consistency
* refactor: Reorganize retrievers exports in langchain codebase
- Reorganize exports in the `index.ts` file in the `retrievers` directory
- Move `MetalRetriever` and `RemoteLangChainRetriever` exports to the top
- Add `TimeWeightedVectorStoreRetriever` export at the bottom for improved organization
* feat: Refactor TimeWeightedVectorStoreRetriever interface and add tests
- Improve time-weighted retriever functionality
- Add JSDoc comments for key retriever functions
- Refactor tests for better isolation and add tests for new functionality
- Expose new interface for retriever in index file
* refactor: Refactor time-weighted retriever constructor and add JSDoc comments
- Simplified the constructor arguments using optional chaining and default values.
- Improved code clarity by adding JSDoc comments for private functions.
- Renamed variable for better understanding of its purpose.
- Removed unnecessary keyword in a method.
* refactor: Refactor test file naming convention.
- Rename `time_weighted3.js` to `time_weighted.js` in the `time_weighted.test.ts` file in `langchain/src/retrievers/tests`.
- No significant changes to other files were made.
* style: Improve code style in retrievers index file
- Add missing comma to SupabaseHybridSearchParams
- Reformat import statements in retrievers/index.ts
- Improve code readability and maintainability in langchain/src/retrievers/
* style: Remove unused eslint-disable comments in test file
- Remove unnecessary eslint-disable comments in the `time_weighted.test.ts` file
- Improve code readability and maintainability
- Enhance overall code quality and consistency
* Lint
* Unod changes to index
* Add entrypoint
* Adds thrown error when using TimeWeightedVectorStoreRetriever on unsupported vector stores, adds docs and test
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Jacob Lee <jacob@autocode.com>
Jacob Lee