## Background
This PR fixes this error when there are special tokens when querying the
chain:
```
Encountered text corresponding to disallowed special token '<|endofprompt|>'.
If you want this text to be encoded as a special token, pass it to `allowed_special`, e.g. `allowed_special={'<|endofprompt|>', ...}`.
If you want this text to be encoded as normal text, disable the check for this token by passing `disallowed_special=(enc.special_tokens_set - {'<|endofprompt|>'})`.
To disable this check for all special tokens, pass `disallowed_special=()`.
```
Refer to the code snippet below, it breaks in the chain line.
```
chain = ConversationalRetrievalChain.from_llm(
ChatOpenAI(openai_api_key=OPENAI_API_KEY),
retriever=vectorstore.as_retriever(),
qa_prompt=prompt,
condense_question_prompt=condense_prompt,
)
answer = chain({"question": f"{question}"})
```
However `ChatOpenAI` class is not accepting `allowed_special` and
`disallowed_special` at the moment so they cannot be passed to the
`encode()` in `get_num_tokens` method to avoid the errors.
## Change
- Add `allowed_special` and `disallowed_special` attributes to
`BaseOpenAI` class.
- Pass in `allowed_special` and `disallowed_special` as arguments of
`encode()` in tiktoken.
---------
Co-authored-by: samcarmen <“carmen.samkahman@gmail.com”>
re
https://github.com/hwchase17/langchain/issues/439#issuecomment-1510442791
I think it's not polite for a library to use the root logger
both of these forms are also used:
```
logger = logging.getLogger(__name__)
logger = logging.getLogger(__file__)
```
I am not sure if there is any reason behind one vs the other? (...I am
guessing maybe just contributed by different people)
it seems to me it'd be better to consistently use
`logging.getLogger(__name__)`
this makes it easier for consumers of the library to set up log
handlers, e.g. for everything with `langchain.` prefix
Note to self: Always run integration tests, even on "that last minute
change you thought would be safe" :)
---------
Co-authored-by: Mike Lambert <mike.lambert@anthropic.com>
* Adds an Anthropic ChatModel
* Factors out common code in our LLMModel and ChatModel
* Supports streaming llm-tokens to the callbacks on a delta basis (until
a future V2 API does that for us)
* Some fixes
When using the llama.cpp together with agent like
zero-shot-react-description, the missing branch will cause the parameter
`stop` left empty, resulting in unexpected output format from the model.
This patch fixes that issue.
RWKV is an RNN with a hidden state that is part of its inference.
However, the model state should not be carried across uses and it's a
bug to do so.
This resets the state for multiple invocations
Added support for passing the openai_organization as an argument, as it
was only supported by the environment variable but openai_api_key was
supported by both environment variables and arguments.
`ChatOpenAI(temperature=0, model_name="gpt-4", openai_api_key="sk-****",
openai_organization="org-****")`
Add support for defining the organization of OpenAI, similarly to what
is done in the reference code below:
```
import os
import openai
openai.organization = os.getenv("OPENAI_ORGANIZATION")
openai.api_key = os.getenv("OPENAI_API_KEY")
```
Add `n_batch` and `last_n_tokens_size` parameters to the LlamaCpp class.
These parameters (epecially `n_batch`) significantly effect performance.
There's also a `verbose` flag that prints system timings on the `Llama`
class but I wasn't sure where to add this as it conflicts with (should
be pulled from?) the LLM base class.
the j1-* models are marked as [Legacy] in the docs and are expected to
be deprecated in 2023-06-01 according to
https://docs.ai21.com/docs/jurassic-1-models-legacy
ensured `tests/integration_tests/llms/test_ai21.py` pass.
empirically observed that `j2-jumbo-instruct` works better the
`j2-jumbo` in various simple agent chains, as also expected given the
prompt templates are mostly zero shot.
Co-authored-by: Michael Gokhman <michaelg@ai21.com>
# What does this PR do?
This PR adds similar to `llms` a SageMaker-powered `embeddings` class.
This is helpful if you want to leverage Hugging Face models on SageMaker
for creating your indexes.
I added a example into the
[docs/modules/indexes/examples/embeddings.ipynb](https://github.com/hwchase17/langchain/compare/master...philschmid:add-sm-embeddings?expand=1#diff-e82629e2894974ec87856aedd769d4bdfe400314b03734f32bee5990bc7e8062)
document. The example currently includes some `_### TEMPORARY: Showing
how to deploy a SageMaker Endpoint from a Hugging Face model ###_ ` code
showing how you can deploy a sentence-transformers to SageMaker and then
run the methods of the embeddings class.
@hwchase17 please let me know if/when i should remove the `_###
TEMPORARY: Showing how to deploy a SageMaker Endpoint from a Hugging
Face model ###_` in the description i linked to a detail blog on how to
deploy a Sentence Transformers so i think we don't need to include those
steps here.
I also reused the `ContentHandlerBase` from
`langchain.llms.sagemaker_endpoint` and changed the output type to `any`
since it is depending on the implementation.
Given that different models have very different latencies and pricings,
it's benefitial to pass the information about the model that generated
the response. Such information allows implementing custom callback
managers and track usage and price per model.
Addresses https://github.com/hwchase17/langchain/issues/1557.
PromptLayer now has support for [several different tracking
features.](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9)
In order to use any of these features you need to have a request id
associated with the request.
In this PR we add a boolean argument called `return_pl_id` which will
add `pl_request_id` to the `generation_info` dictionary associated with
a generation.
We also updated the relevant documentation.
Solves https://github.com/hwchase17/langchain/issues/1412
Currently `OpenAIChat` inherits the way it calculates the number of
tokens, `get_num_token`, from `BaseLLM`.
In the other hand `OpenAI` inherits from `BaseOpenAI`.
`BaseOpenAI` and `BaseLLM` uses different methodologies for doing this.
The first relies on `tiktoken` while the second on `GPT2TokenizerFast`.
The motivation of this PR is:
1. Bring consistency about the way of calculating number of tokens
`get_num_token` to the `OpenAI` family, regardless of `Chat` vs `non
Chat` scenarios.
2. Give preference to the `tiktoken` method as it's serverless friendly.
It doesn't require downloading models which might make it incompatible
with `readonly` filesystems.
Carmen Sam