langchain-python

mirror of https://github.com/Mintplex-Labs/langchain-python.git synced 2026-07-21 00:35:23 -04:00

Files

T

Kyle Roth c7db9febb0 count tokens for new OpenAI model versions (#6195 )

Trying to call `ChatOpenAI.get_num_tokens_from_messages` returns the
following error for the newly announced models `gpt-3.5-turbo-0613` and
`gpt-4-0613`:

```
NotImplementedError: get_num_tokens_from_messages() is not presently implemented for model gpt-3.5-turbo-0613.See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens.
```

This adds support for counting tokens for those models, by counting
tokens the same way they're counted for the previous versions of
`gpt-3.5-turbo` and `gpt-4`.

#### reviewers

  - @hwchase17
  - @agola11

2023-06-15 06:16:03 -07:00

__init__.py

Harrison/vertex (#5049 )

2023-05-24 15:51:12 -07:00

anthropic.py

Enable serialization for anthropic (#6049 )

2023-06-12 22:39:10 -07:00

azure_openai.py

WIP: openai settings (#5792 )

2023-06-06 21:57:58 -07:00

base.py

propogate kwargs fully (#6076 )