Commit Graph

350 Commits

Author SHA1 Message Date
jacoblee93 6228b6fdce Adds separate page for custom tools 2023-05-27 15:57:03 -07:00
DanielPopOut fbfb4f6103 Remove outdated remark in docs/modules/models/chat/index.mdx
According to the docs there is more than ChatOpenAi model only --> ChatAnthropic etc...

What was removed:
"Note: Currently, the only chat-based model we support is ChatOpenAI (with gpt-4 and gpt-3.5-turbo), but anticipate adding more in the future."
2023-05-26 18:22:24 -04:00
Jacob Lee dc948fce4f Tigris vectorstore (#1423)
* Add Tigris Vector Store

* review feedback

* Skip integration test, small docs update

* Remove unused example

* Revert build artifact change

* Remove build artifact

---------

Co-authored-by: Ovais Tariq <ot@tigrisdata.com>
Co-authored-by: Ovais Tariq <ovaistariq@gmail.com>
2023-05-26 12:52:36 -07:00
Orgil da6320ac3c feat: notion db loader (#1342)
* wip notion db loader

* add notiondb entrypoint

* add notiodb files to .gitignore

* add notiondb to tsconfig

* add notiondb doc

* add notiondb test and example

* update entrypoint

* little refactor

* rename notiondb.md to notiondb.mdx

* add NOTION_INTEGRATION_TOKEN env

* add page size configureable and read token from env

* make params optional

* update init notion db loader

* Change NotionDB unit test to integration test, update docs, polish

* Fix typo in docs

* Fix typo

* Use new environment variable method

* Update docs

* Merge

* Merge

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>
2023-05-26 12:35:53 -07:00
Steven Tsao 96038e3926 Add latex text splitter (#1127)
* Add latex text splitter

* update

* Correct tests

* Add docs and example for LatexTextSplitter

* Lint + format

* Fix test

* remove log

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>
2023-05-26 12:04:49 -07:00
Orgil a7dfabdab1 feat: add figma loader (#1148)
* wip figma file loader

* add figma env to .env.sample

* update FigmaFile ts

* fix format

* add figma to entry-points

* add figma doc and example

* include figma files in .gitignore

* add figma to tsconfig.json

* little refactor

* rename md to mdx

* add FIGMA_ACCESS_TOKEN to .env.example

* update figma example and int test

* Update Figma loader args, test case, examples, and docs

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>
2023-05-26 12:02:17 -07:00
Fan 70ac7c41e6 fix: #1383 update faiss-node to avoid libomp missing (#1420) 2023-05-26 11:36:58 -07:00
Jacob Lee 72a40526c8 Google Vertex AI docs update 2023-05-24 17:33:13 -07:00
Jacob Lee ded81a3439 Adds Qdrant vector store (#1396)
* feat: add qdrant vertorstore

* feat: add test case

* chore: update package.json

* update qdrant config

* Fix bugs, naming issues, use examples in docs

* Adds Qdrant as a dependency in examples

* Update docs, remove unneeded dep in examples

---------

Co-authored-by: ppxu <xyang0110@126.com>
2023-05-24 16:53:52 -07:00
Allen Firstenberg 2b6e328077 Feature: Google Vertex AI Chat model (#1384)
* Google Vertex AI Chat model

* Simplify Google Vertex AI chat class, use more LangChain conventions

* Adds chat_models/googlevertexai to optional dependencies

* Fix example, fix build

* Clarify docs

* Formatting

* Docs styling

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-24 16:45:34 -07:00
Jacob Lee 1f787a6ec5 Add API chain (#1400)
* adding initial API chain code

* lint fix

* add docs and example

* Update file structure, move around parameters and TS types

* Slightly shorten APIChain example for consistency

---------

Co-authored-by: Larry Anderson <larryboymi@hotmail.com>
2023-05-24 16:39:34 -07:00
Henrique Dantas 052664c1b1 feat: Created Upstash Redis Chat Memory for Edge and Serverless environments (#1346)
* installed @upstash/redis

* updating .env.example with upstash redis vars

* feat: created RedisUpstashChatMessageHistory based on RedisChatMessageHistory

* test: adding tests based on redis tests

* docs: adding docs with examples for upstash redis

* adding @upstash/redis as examples dependency

* adding stores/message/redis_upstash to langchain .gitignore

* adding stores/message/redis_upstash to langchain package.json files, peerDependenciesMeta and exports

* adding stores/message/redis_upstash to langchain tsconfig.json

* adding stores/message/redis_upstash to create-entrypoint.js

* adding @upstash/redis to examples yarn.lock

* moving @upstash/redis peerDependenciesMeta to correct order

* updating main yarn.lock with @upstash/redis

* lint: adding missing semicolon to import

* adding eslint-disable lines to redis_upstash int test file

* moving typedocOptions redis_upstash to the correct place

* docs: improvements to redis_upstash docs

* Change Upstash Redis naming, update docs

* Fix Upstash Redis constructor config types

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-24 16:30:52 -07:00
yi 7fec78b0ec Added google custom search as a tool (#1260)
* Added google custom search as a tool

* Google Custom Search API tool docs, test, small parameter change

* Make Google custom search tool read from environment variables if possible, change param names

* Fix typo

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-24 16:21:35 -07:00
Allen Firstenberg dc7bfd80b6 Feature: Google Vertex AI Embeddings model (#1388)
* Fix type definition of an array

* Impement GoogleVertexAIEmbeddings

* GoogleVertexAIEmbeddings documentation and examples

* Create entrypoints, minor renaming and docs update

* Make response nullable for more graceful failures

* Format

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-24 16:07:44 -07:00
Seunghun Shin c661eec7e9 Update index.md (#1397) 2023-05-24 11:09:39 -07:00
Jacob Lee d5cee57473 Add docs on hidden faiss-node dependency, change unit test to an integration test (#1385)
* Add docs on hidden faiss-node dependency, change unit test to an integration test

* Switch comment to the correct test

* Fix integration test
2023-05-23 17:55:25 -07:00
Romeo 1290d4fe7c Feature: Enhanced Document Splitting with Contextual Chunk Headers (#961)
* improvement: add the option to add headers to chunks so to help LLMs and vector stores better understand linked chunks

* docs: update text splitters docs to reference `chunkHeader`, `chunkOverlapHeader`, and `showChunkOverlapHeader`

* fix: formatting

* Move contextual headers for each chunk out of the TextSplitter constructor and into individual methods

* Fix typo

* Adds example result

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-23 10:34:40 -07:00
Allen Firstenberg 5e42176eef Feature: Google Vertex AI PaLM Support (#1273)
* Initial work implementing access to the Google Vertex AI API and the LLM models available through it.

* Refactor into a base class

* Prettier

* Add chat version of Google Vertex AI LLM.

* Add Google Vertex AI classes

* Example for Google Vertex AI LLM

* Add Google Vertex AI LLM

* Documentation

* Google Vertex AI Chat example

* Fix capitalization typo

* Make Google's SDK a peer dependency, move out of main entrypoint

* Add convenience method for type equality.

* Extensive refactoring to make the Google Vertex AI Chat implementation compliant

* Legacy prettier cleanup

* Prettier cleanup of refactored code

* Refactor to move connection and common types into separate modules.

* Refactor to move connection and common types into separate modules.

* Refactor to move connection and common types into separate modules.

* Change maxTokens to maxOutputTokens.
Change configuration for the connection object.
Prettier

* ChatGoogleVertexAi entry point

* GoogleVertexAi tests

* GoogleVertexAi example refactoring

* Example for ChatGoogleVertexAi
Fix and test for determining context and examples.

* Documentation for Google VertexAI text and chat models.

* Refactor some of ChatGoogleVertexAi to break out some components to be better unit tested.
Add unit tests.
Fix bugs with context and example generation.
Formatting.

* Descope PR to just GoogleVertexAI LLM, normalize class names

* Small fixes and renames, increase default max token output size

* Fix typo in documentation import from examples. I think.

* Fix docs

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-23 10:06:08 -07:00
Tõnis Tiganik 69a691b778 ignore like .gitignore in GithubLoader (#1177)
* ignore like .gitignore in GithubLoader

* set ignore as optional depenency

* Make ignore an optional import for GithubRepoLoader, refactor shouldIgnore method

* Formatting

* Remove conditional import, remove GitHubRepoLoader from index.ts file

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-23 09:58:00 -07:00
Jacob Lee 4ca0b5f400 Merge pull request #1352 from kristofdombi/patch-1
[DOCS]: Add Kinsta to list of providers
2023-05-22 17:58:32 -07:00
Jacob Lee 9c57020e87 Update document_qa.mdx 2023-05-22 08:43:46 -07:00
Or Gal 3eac137397 chore: fixing docs 2023-05-22 15:34:40 +03:00
Or Gal 11c197fa1a chore: fixing docs 2023-05-22 15:23:21 +03:00
Or Gal 7ca5a891b4 chore: fixing docs 2023-05-22 15:09:54 +03:00
Kristóf Dombi 322c51afca docs(deployment): add kinsta to list of providers 2023-05-21 16:07:42 +02:00
Max Adam 0dc396d44f Update motorhead_memory.md
motorheadUrl -> motorheadURL 

Assuming this is what was wanted, motorheadURL is the correct property name on class MotorheadMemory.
2023-05-20 20:47:38 +10:00
Nicolas 6fcd138c35 Nick: v102 (#1326) 2023-05-19 15:51:56 +01:00
Jacob Lee c8d4e6bf7e Adds structured tool chat agent and DynamicStructuredTool class (#1103)
* [WIP] Structured tool input agent

* Clean up typing

* Update StructuredChatAgent prompt, adds integration test and validation preventing StructuredTools being passed into existing agents

* Undo change to make existing agents less strictly typed, use overload signatures for initialize

* Update parsing logic for retries

* Update test cases

* Progress on structured chat agent

* Pass tool names into structured chat output fixing parser for more reliability

* Revert output parser getFormatInstructions method signature change

* Adds StructuredDynamicTool, docs

* add renderTemplate call to format instructions, change StructuredDynamicTool to DynamicStructuredTool

* Formatting

---------

Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>
Co-authored-by: Jacob Lee <jacob@autocode.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-05-19 15:48:09 +01:00
Fan c92ba9beaa feat: add Faiss as VectorStore provider (#685)
* add faiss

* add pickle support

* replace pickle.js with pickleparser package

* yarn install

* update pickleparser

* fix code review

* fix metadata in loadFromPython

* add docs

* add test cases

* port  form python

* Save mapping into JSON file.

* upgrader pickleparser

* upgrade pickleparser

* make loadFromPython work

* upgrade pickleparser

* upgrade pickleparser and use reduce function

* update demo and docs

* fix doc build

* update docs

* add more ut and fix import of pickleparser

* update docs

* add int test for loadFromPython

* add more int test for loadFromPython

* fix merge error

* upgrade pickleparser

* upgrade pickleparser and fix code review

* yarn install

* port #1142

* use chained register

* fix lint

* fix format

* Update examples, update imports, remove extraneous deps from merge

* Undo change to imports

---------

Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-05-18 18:07:30 +01:00
Nuno Campos 0cc685e5d0 Update sagemaker doc 2023-05-18 16:28:31 +01:00
Nuno Campos 603f243ee0 Add verbose env var (#1317) 2023-05-18 15:15:12 +01:00
David Duong 0f62af5026 Add option to return used SQL in SqlDatabaseChain (#1313)
* Add option to return used SQL in SqlDatabaseChain

* Cleanup examples
2023-05-18 13:46:36 +01:00
Priya X. Pramesi 74e49883bb New feat: Self Query Retriever (#1266)
* self-query with meriyah

* removed builtin translators so end-user needs to specify their own translator, but a basic translator that can be used with pinecone and chroma is included and can be imported

* fixed class name typo

* moved meriyah to main dependency so expression parser can work in test-exports-*

* added back self_query to entrypoints

* moved meriyah to dev/peer dep, fixed unused parser types, minor fixes

* renaming expression type to reflect meriyah better

* removed unused lint disable on self_query/base.ts
2023-05-18 13:11:43 +01:00
Christopher M 6f6d60a642 feat: Add sagemaker endpoint support (#1267)
* feat: Add sagemaker endpoint support

* fix: Add missing export for SagemakerEndpoint

* fix: Align kwargs types with codebase for sagemaker endpoint

* chore: Add integration test for sagemaker_endpoint

* feat: Add docs for sagemaker endpoint llm

* Update SageMaker endpoint args, integration test, imports, docs

* Fix typo

* Use async caller

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-05-18 13:11:23 +01:00
Nuno Campos 6dc56f8d0a Update vectorstore toolkit example (#1297)
* Update vectorstore toolkit example

* Improve docs for supabase metadata filtering

* Update hnswlib examples

* Fix links
2023-05-17 20:40:24 +01:00
David Duong d60eae5995 Use a JS based tokenizer for token counting (#1239)
* Use a rewritten tiktoken for token counting

* Fix lint, add test

* Fix type import, typo

* Remove @dqbd/tiktoken as "weak" dependency

* Move JS implementation of tiktoken into a separate package

* Upgrade to js-tiktoken@1.0.2

* Replace all occurrences of @dqbd/tiktoken

* Avoid bundling ranks, fix test with CRA

* Use AsyncCaller, cache promises

* Remove WASM configuration changes

* Upgrade to 1.0.6 for CRA fixes, mark js-tiktoken/lite as an external dependency

* Remove next.config.js
2023-05-17 14:44:29 +01:00
Jacob Lee dc5dee0631 Merge pull request #1285 from dhruv-anand-aintech/dhruv
fix: typo (missing word "simple")
2023-05-16 17:53:41 -07:00
Jacob Lee baa4c9cf43 Update api.mdx 2023-05-16 14:39:42 -07:00
Dhruv Anand 6cb3a7e147 fix: typo (missing word "simple") 2023-05-16 22:54:41 +05:30
RohitMidha23 be3f5cbad1 feat: entity memory (#680)
* init entity_memory

* init tests

* added test

* fixed test

* updated prompts usage

* fixed import

* added test for pre loaded messages

* updated import

* Use object without prototype for safer exists check

* move entity store to its own file, make it async

* Add example, not giving great results for now

* Prompt updates to support reflexive pronouns, keep entity summaries unchanged, support chat models, add docs

* Remove merge artifact

* Fix test

* Format fix

---------

Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-16 18:24:35 +01:00
mvaker fc7fc868aa Add Redis VectorStore (#1153)
* feat: add Redis vectorstore with tests

* fix linting and formatting problems

* docs: add redis vector store

* fix: remove export from deprecated vectorstores index

* docs: add node.js-only compatibility tip to redis vectorstore

* Update types, add more optional config, move examples out of md file

* Use more recent chain in example

* Update tests

* Mark entrypoint as optional dep

* Fix import

* Fix build error

---------

Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-05-16 18:24:09 +01:00
Jacob Lee 1ed96d124a Merge pull request #1271 from hwchase17/jacoblee/apify
Add Apify integration, update docs
2023-05-15 13:48:09 -07:00
Jacob Lee 6b4b2ca454 Adds examples and docs 2023-05-15 11:52:08 -07:00
Jacob Lee ce5042dd74 Remove ApifyWrapper references 2023-05-15 09:51:33 -07:00
Amijakan 3300abe340 Fixed typo with example
includeTables => includesTables
2023-05-15 10:21:09 -04:00
František Nesveda 32816c04d2 Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	docs/docs/modules/agents/tools/integrations/index.mdx
#	langchain/package.json
#	yarn.lock
2023-05-15 14:48:33 +02:00
Chris Toomey ace3ee7029 Adding Redis Memory Support (#951)
* Merge post upstream sync (#1)

* Added RedisMemory with tests and docs

* fixed Redis test file name

* TODOs for quick refactor

* updated memory to accept a client

* Added TODOS

* ongoing testing

* Updated tests

* Updated tests for memory return option

* finalized tests + updated docs

* Adding docs

* Readded init and cleaned up func

* fixed test typing

* redo yarn lock

* remove yarn.lock

* updated yarn lock and namespaced redis

* fix merge conflict

* updated BaseChatMemoryInput

* Updated with lint fixes

* Fixed docs to match memory instantiation

* yarn format docs

* Merging fixes to address ForkPR comments (#2)

* updating for pr

* Removed redis_memory in favor of chat memory

* Fixed tests and updated docs

* Bump docs

* lint results

* fixes from lint and format

---------

Co-authored-by: Chris Toomey <ctoomey@fb.com>

* Update RedisChatMessageHistory class to initialize a client internally

* Update Redis chat message history docs to reflect the fact that Redis is a peer dependency, allow direct passing of Redis client

* Patch typing issues with passing node-redis client directly into a RedisChatMessageHistory instance

---------

Co-authored-by: Chris Toomey <ctoomey@fb.com>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-11 17:14:21 +01:00
Pranav Malvawala dad9643c97 fix: typo in schema docs (#1198) 2023-05-11 12:22:11 +01:00
Nuno Campos 7fa918188a Implement cancellation and timeout for all LLMs, allow passing LLM call options via LLMChain (like was already possible for stop) (#1182)
* Implement cancellation and timeout for all LLMs, allow passing LLM call options via LLMChain (like was already possible for stop)

* Update example

* Add declared ParsedCallOptions type for BaseLanguageModels, add signal support for chat models, add tests

* Update docs for chat models

* Update llm and chat examples to remove `run` func

* Add more examples for LLMChain

* Reorganise agent docs, add Additional Functionality doc for Agents

* Add toc in llm chain docs

* Format

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
2023-05-11 12:21:48 +01:00
Nuno Campos 8200eff813 Merge pull request #1199 from jacoblee93/feature_plan_and_execute
Adds plan-and-execute style agent
2023-05-10 16:31:27 +01:00