mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
Abby Harrison
b2e1548379
### Motivation and Context To take advantage of APIs offered by most LLMs and to be in sync with .NET SK, this PR introduces the ability to generate multiple text completions or chat completions from a single prompt.  ### Description - Return type hint for `complete_async` and `complete_chat_async` changed from `str -> Union[str, List[str]]`. The use of `Union` is the proper way to indicate multiple return types prior to Python 3.10+. 3.10 supports the use of the `|` symbol, but since the Python SK is supported on 3.8 and 3.9, I did not adopt the newer standard. - `complete_async`, `complete_stream_async`, `complete_chat_async`, and `complete_chat_stream_async` now support settings field `number_of_responses` greater than 1. Previously only a value of 1 was supported. - **Note: hf_text_completion does not support streaming multiple responses due to a limitation of TextIteratorStreamer. This feature requires the ability to parse distinct responses from TextIteratorStreamer.** - Fixed a bug where `complete_async` was streaming single responses as 1D arrays, content now is simply a string. ### Example Usage #### Setup ``` kernel = sk.Kernel() # Configure OpenAI service api_key, org_id = sk.openai_settings_from_dot_env() oai_text_service = OpenAITextCompletion("text-davinci-003", api_key, org_id) oai_chat_service = OpenAIChatCompletion("gpt-3.5-turbo", api_key, org_id) # Configure Hugging Face service hf_text_service = HuggingFaceTextCompletion("gpt2", task="text-generation") # Configure Prompt prompt = "what is the purpose of a rubber duck?" # Configure Request Settings text_request_settings_multi = CompleteRequestSettings( max_tokens=100, temperature=0.7, top_p=1, frequency_penalty=0.5, presence_penalty=0.8, number_of_responses=4 ) chat_request_settings_multi = ChatRequestSettings( max_tokens=100, temperature=0.7, top_p=1, frequency_penalty=0.5, presence_penalty=0.8, number_of_responses=4 ) ``` #### Text Completion (Standard) ``` texts = await oai_text_service.complete_async(prompt, text_request_settings_multi) i = 0 for text in texts: print("Option " + str(i) + ": " + text) i += 1 ``` #### Streaming Text Completion ``` multi_stream = oai_text_service.complete_stream_async(prompt, text_request_settings_multi) texts = [''] * text_request_settings_multi.number_of_responses async for text in multi_stream: i = 0 os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience print("PROMPT: " + prompt) for option in text: texts[i] = texts[i] + option print("{0}: {1}".format(i, texts[i])) i += 1 ``` #### Chat Completion (Standard) ``` texts = await oai_chat_service.complete_chat_async([("user",prompt)], chat_request_settings_multi) i = 0 for text in texts: print("Option " + str(i) + ": " + text) i += 1 ``` #### Streaming Chat Completion ``` multi_stream = oai_chat_service.complete_chat_stream_async([("user",prompt)], chat_request_settings_multi) texts = [''] * chat_request_settings_multi.number_of_responses async for text in multi_stream: i = 0 os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience print("PROMPT: " + prompt) for option in text: texts[i] = texts[i] + option print("{0}: {1}".format(i, texts[i])) i += 1 ``` #### HuggingFace Standard Completion ``` texts = await hf_text_service.complete_async(prompt, request_settings_multi) i = 0 for text in texts: print("-----------------------------------") print("Option " + str(i) + ": " + text) i += 1 ```