mirror of https://github.com/microsoft/semantic-kernel.git synced 2026-10-12 02:19:06 +00:00

Files

T

Abby Harrison f6059cdf94 Python/Local Hugging Face Inference for Completions and Embeddings (#658 )

### Motivation and Context
This PR introduces native python support for Hugging Face models that
can: complete text, generate new text, summarize, and that generate
embeddings. Currently only supports downloading the models locally from
the HF model hub. Future plans include supporting the HF inference API
as well.

### Description
- Added 2 services: `hf_text_completion` and `hf_text_embedding`
- `hf_text_completion` supports the following tasks: _text-generation_,
_text2text-generation_, and _summarization_
- `hf_text_embedding` supports any model supported by the
sentence-transformers pip package
- Added dependencies: pytorch, transformers, sentence-transformers to
`requirements.txt` and `poetry.lock`
- fixed typo: `get_embedding_service_service_id` ->
`get_embedding_service_id`
- Added a number of integration tests for supported HF models

2023-04-26 17:07:29 -07:00

3.2 KiB

Raw Blame History

Kernel Feature Matrix by Language

AI Services

	C#	Python	Notes
TextGeneration	✅	✅	Example: Text-Davinci-003
TextEmbeddings	✅	✅	Example: Text-Embeddings-Ada-002
ChatCompletion	✅	✅	Example: GPT4, Chat-GPT
Image Generation	✅	❌	Example: Dall-E

AI Service Endpoints

	C#	Python	Notes
OpenAI	✅	✅
AzureOpenAI	✅	✅
Hugging Face Inference API	🔄	❌	Coming soon to Python, not all scenarios are covered for .NET
Hugging Face Local	❌	✅
Custom	✅	🔄	Requires the user to define the service schema in their application

Tokenizers

	C#	Python	Notes
GPT2	✅	✅
GPT3	✅	❌
tiktoken	🔄	❌	Coming soon to Python and C#. Can be manually added to Python via `pip install tiktoken`

Core Skills

	C#	Python
TextMemorySkill	✅	✅
ConversationSummarySkill	✅	❌
FileIOSkill	✅	✅
HttpSkill	✅	✅
MathSkill	✅	❌
TextSkill	✅	✅
TimeSkill	✅	✅

Planning

	C#	Python	Notes
Plan	✅	❌
SequentialPlanner	✅	❌

Connectors and Skill Libraries

	C#	Python	Notes
Qdrant (Memory)	✅	❌	Vector optimized
ChromaDb (Memory)	❌	🔄
Milvus (Memory)	❌	❌	Vector optimized
Pinecone (Memory)	❌	❌	Vector optimized
Weaviate (Memory)	❌	❌	Vector optimized
CosmosDB (Memory)	✅	❌	CosmosDB is not optimized for vector storage
Sqlite (Memory)	✅	❌	Sqlite is not optimized for vector storage
Azure Cognitive Search	❌	❌
MsGraph	✅	❌	Contains connectors for OneDrive, Outlook, ToDos, and Organization Hierarchies
Document Skills	✅	❌	Currently only supports Word documents
OpenAPI	✅	❌
Web Skills	✅	❌

Design Choices

The overall architecture of the core kernel is consistent across Python and C#, however, the code should follow common paradigms and style of each language.

During the initial development phase, many Python best practices have been ignored in the interest of velocity and feature parity. The project is now going through a refactoring exercise to increase code quality.

To make the Kernel as lightweight as possible, the core pip package should have a minimal set of external dependencies. On the other hand, the SDK should not reinvent mature solutions already available, unless of major concerns.

3.2 KiB Raw Blame History