Abby Harrison f6059cdf94 Python/Local Hugging Face Inference for Completions and Embeddings (#658) ...

### Motivation and Context
This PR introduces native python support for Hugging Face models that
can: complete text, generate new text, summarize, and that generate
embeddings. Currently only supports downloading the models locally from
the HF model hub. Future plans include supporting the HF inference API
as well.

### Description
- Added 2 services: `hf_text_completion` and `hf_text_embedding`
- `hf_text_completion` supports the following tasks: _text-generation_,
_text2text-generation_, and _summarization_
- `hf_text_embedding` supports any model supported by the
sentence-transformers pip package
- Added dependencies: pytorch, transformers, sentence-transformers to
`requirements.txt` and `poetry.lock`
- fixed typo: `get_embedding_service_service_id` ->
`get_embedding_service_id`
- Added a number of integration tests for supported HF models

2023-04-26 17:07:29 -07:00

..

__init__.py

Python/Local Hugging Face Inference for Completions and Embeddings (#658)

2023-04-26 17:07:29 -07:00

function_extension.py

Python: Text partitioning module (#450)

2023-04-26 22:50:24 +00:00

text_chunker.py

Python: Text partitioning module (#450)

2023-04-26 22:50:24 +00:00