Lee Miller
5b1ed2f192
### Motivation and Context To match changes in #450 ### Description This pull request renames the SemanticTextPartitioner class to TextChunker, and updates all references to this class in the code and documentation. The TextChunker class is responsible for splitting text into smaller chunks based on tokens and paragraphs, which is useful for processing large texts or code files. The new name TextChunker reflects the functionality of the class more clearly and avoids confusion with the SemanticPartitioner class, which is a different concept that splits text into semantic segments based on embeddings. Details: - Rename SemanticTextPartitioner.cs to TextChunker.cs and update the class name and namespace accordingly. The TextChunker class is now in the Text namespace, which is more consistent with its purpose and the other classes in the namespace. - Update all references to SemanticTextPartitioner in the code and documentation to use TextChunker instead. This includes the ConversationSummarySkill and the DocumentImportController classes, which use the TextChunker class to split text into sentences and paragraphs for summarization and import. - Update the README.md file for the GitHub Repo Q&A Bot sample to explain how the TextChunker class works with memory and embeddings. - Make some minor formatting changes to the TextChunker class, such as adding line breaks and spaces for readability, and removing an unused using directive.
2.3 KiB
GitHub Repo Q&A Bot Sample
!IMPORTANT This learning sample is for educational purposes only and should not be used in any production use case. It is intended to highlight concepts of Semantic Kernel and not any architectural / security design practices to be used.
Running the sample
- You will need an Open AI Key or Azure Open AI Service key for this sample.
- Ensure the service API is already running
http://localhost:7071. If not, learn how to start it here. - You will also need to Copy .env.example into a new file with name ".env".
- Run the following command
yarn install(if you have never run the sample before) and/oryarn startfrom the command line. - A browser will open or you can navigate to
http://localhost:3000to use the sample.
Working with Secrets: KernelHttpServer's Readme has a note on safely working with keys and other secrets.
About the GitHub Repo Q&A Bot Sample
The GitHub Repo Q&A Bot sample allows you to pull in data from a public GitHub repo into a local memory store in order to ask questions about the project and to get answers about it. The sample highlights how memory and embeddings work along with the TextChunker when the size of the data is larger than the allowed token limited. Each SK function will call Open AI to perform the tasks you ask about.
In order to reduce costs and improve overall performance, this sample app indexes only content extracted from markdown files.
Caution
Each function will call Open AI which will use tokens that you will be billed for.
Next Steps
Create Skills and SK functions: Check out the documentation for how to create Skills.
Join the community: Join our Discord community to share ideas and get help.
Contribute: We need your help to make this the best it can be. Learn how you can contribute to this project.