### Motivation and Context
Implement Azure OpenAI Dall E
### Description
Implement Azure OpenAI Image Generation according to the official
[documentation](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#image-generation)
and the examples provided by Azure OpenAI Studio, so that it can be used
in SK.
Due to the differences between the API of Azure OpenAI Dall-E and OpenAI
Dall-E, some modifications have been made to OpenAIClientBase under
CustomClient to ensure compatibility.
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
### Motivation and Context
This pull request fixes a bug in the sequential planner example and
improves the custom planner example by adding input reset and context
update steps. These changes ensure that the input and context are
consistent and up-to-date for each step of the plan execution.
### Description
- In the sequential planner example, add an input reset step after each
plan step execution. This prevents the input from accumulating and
interfering with the subsequent steps.
- In the custom planner example, update the context variables with the
input before invoking the plan. This ensures that the context query
skill can access the input as a variable and perform the appropriate
queries.
This commit adds a new option to the default section of the typos.toml
file, which allows users to specify a list of regular expressions that
match identifiers that should be ignored by the typos checker. This can
be useful for cases where some identifiers are intentionally spelled in
a non-standard way, or contain abbreviations or acronyms that are not
recognized by the dictionary.
One example of such a case is the Azure Graph Service, which uses the
abbreviation "ags" in some of its identifiers. To avoid false positives
from the typos checker, we can add "ags" to the extend-identifiers
section of the typos.toml file. However, this would also ignore any
other occurrences of "ags" in the codebase, which may not be desirable.
To address this issue, we can use the extend-ignore-identifiers-re
option to specify a regular expression that matches only the "ags"
identifiers that are part of the Azure Graph Service API, such as
"2021-03-01_ags". This way, we can fine-tune the typos checker to ignore
only the identifiers that we want, and still catch any other potential
spelling errors.
### Motivation and Context
Other Semantic Kernal samples allow using both OpenAI and Azure OpenAI
by setting a different set of environment variables and toggling the
`useAzureOpenAI` bool value
### Description
Add the same snippet used in other notebooks such as
`02-running-prompts-from-file` to use Azure OpenAI to the
`08-native-function-inline`
SemanticKernel only uses ILogger and extension methods off of it. We
should just be depending on Microsoft.Extensions.Logging.Abstractions,
and not Microsoft.Extensions.Logging. The Logging package brings a lot
more dependencies with it.
cc @stephentoub
Removed embeddingSpan instance in the GetNearestMatchesAsync method,
within the VolatileMemoryStore class, as it's not used in the cosine
similarity calculation.
### Motivation and Context
There are different sets of parameters when using OpenAI vs Azure OpenAI
#### OpenAI
1. OpenAI Key
1. Model (`text-davinci-003`)
#### Azure OpenAI
You can deploy multiple versions of the same model (`gpt-3.5-turbo`)
with different resource throughput. This is why "model" is not unique
enough. Instead of Model you specify Deployment and Endpoint
1. OpenAI Key
1. Deployment
1. Endpoint
The `samples\apps\chat-summary-webapp-react` it was using Model term
which may be misleading
### Description
- samples\apps\chat-summary-webapp-react
- Change label for Deployment form input from Model to Deployment
Bumps [PdfPig](https://github.com/UglyToad/PdfPig) from
0.1.8-alpha-20230423-3898f to 0.1.8-alpha-20230529-6daa2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/UglyToad/PdfPig/releases">PdfPig's
releases</a>.</em></p>
<blockquote>
<h2>0.1.2 Second Alpha</h2>
<p>Adds letter font details and a couple of other bugfixes to the alpha
version.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/UglyToad/PdfPig/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adrian Bonar <[email protected]>
### Motivation and Context
Add potential for IChatCompletion with multiple results per prompt.
### Description
- Adds new Interfaces/Abstractions for Chat Results: `IChatResult`,
`IChatStreamingResult`, `IChatMessage`
- Adds new `IChatCompletion` interface methods for Streaming and
Non-Streaming results
- Moves old `IChatCompletion` interface methods
`IAsyncEnumerable<string> GenerateMessageStreamAsync` and `Task<string>
GenerateMessageAsync` as extensions for retro compatibility
- Changed `ChatAsText` to use the new multiple results implementations
- Makes `ChatHistory.ChatMessage` & `ChatHistory.AuthorsRole` types
obsolete (discouraging usage of those implementation in favor of new
`IChatMessage`
- `ChatHistory` abstraction now extends from `List<IChatMessage>`
- Updates all Samples using the new interfaces.
### Paves the way to ... (Futures)
- Remove `ChatHistory.ChatMessage` inner class
- Remove `ChatHistory.AuthorsRole` enums
- Make `ChatHistory` an abstract class or an `IChatHistory` interface.
- Rename `ITextCompletionResult` to `ITextResult`
- Rename `ITextComplet
### Motivation and Context
This changes our parsing of the .env file to use
[`python-dotenv`](https://pypi.org/project/python-dotenv/) which will be
more flexible.
Tested that the notebooks would still run correctly.
### Motivation and Context
This PR ensures that pyproject.toml with requirements.txt are in sync.
### Description
- removed the `^` from pyproject.toml linting dependencies
- Added >= to requirements.txt
### Motivation and Context
Closes#1271 and addresses several TODOs in ChromaMemoryStore
### Description
- added field `additional_metadata` to `MemoryRecord` and all downstream
methods/classes (see `SemanticTextMemoryBase`, `SemanticTextMemory`,
`MemoryQueryResult`, and `ChromaMemoryStore`)
- addressed ChromaMemoryStore TODO 'what is key used for?' For chromadb,
key and id are effectively the same value, but the two fields are
differentiated for memory stores like Qdrant which require that key be a
uuid. I updated the logic so that this is more apparent. Note that this
results in no change in end-user functionality.
- Updated ChromaMemoryStore TODO 'get_async say embedding is optional
but Record constructor requires it.` MemoryRecord constructor no longer
requires an embedding. Logic is updated and a test added to show that
embedding retrieval is functionally optional.
### Motivation and Context
Common feature request and missing capability to acquire the actual
details from the models we are integrating.
Resolves#802Resolves#618
Resolves Partially #351
Getting the results alternative to #693
### Description
This change introduces a new Property in `object?
SKContext.LastPromptResults` which will contain a list of each result
detail for a given prompt.
The Connector namespace will also provide a Extension methods that
converts this object in the actual class that you will be using to
serialize or get the detailed information about the Model result.
Normal suggested usage
```
var textResult = await excuseFunction.InvokeAsync("I missed the F1 final race");
```
Getting model result as json:
```
var modelResultJson = JsonSerializer.Serialize(textResult.LastPromptResults);
```
Getting model result as a traversable object (using a connector
extension):
```
var modelResultJson = textResult.GetOpenAILastPromptResult()?.Usage.TotalTokens;
```
### Description
This PR introduces the following changes:
1. The PineconeClient class accepts a custom HTTP client, allowing the
hosting application/client code to provide their own instances.
2. A new extension method, `WithPineconeMemoryStore`, has been added to
the `KernelBuilder` class. This method simplifies the registration of
the Pinecone memory store, aligns the registration process with other
connector registrations, and enforces the usage of an HTTP client and
retry handlers configured in the SK SDK.
3. The global static `HttpHandlers` class is removed because it goes
against the agreed approach for the SK SDK's HTTP stack that assumes the
usage of a custom HTTP client for fine-tuning instead of relying on
global static variables.
Co-authored-by: Dmytro Struk <[email protected]>
### Description
This PR introduces the following changes:
1. The classes HuggingFaceTextCompletion and
HuggingFaceTextEmbeddingGeneration now accept a custom HTTP client,
allowing the hosting application/client code to provide their own
instances.
2. Two new extension methods, WithHuggingFaceTextCompletionService and
WithHuggingFaceTextEmbeddingGenerationService, are added to the
KernelBuilder class. These methods simplify the registration of Hugging
Face AI services, align the registration process with other connector
registration, and enforce the usage of an HTTP client and retry handlers
configured by the SK SDK.
3. Several older constructors of the classes are marked as obsolete to
prevent any potential breaking changes.
### Motivation and Context
this PR adds a Files tab to Copilot Chat that allows users to view the
documents that have been uploaded to the Bot. in the future we will
ideally have more supported file types on this list with Word/Office
support being next on my list.
### Preview
<img width="452" alt="image"
src="https://github.com/microsoft/semantic-kernel/assets/52973358/1207541b-be73-4ee7-b488-d5027f169c77">
### Description
changes:
- adds a `MemorySource` model that represents the uploaded document
- changes view in `ChatWindow` from `ChatRoom` to the new
`ChatResourceList` depending on the selected tab
- changes `UpdateAsync()` to `UpsertAsync()` in `IStorageContext`.
previously, `CosmosDbContext` was
using upsert while the other two were updating, whereas now we upsert
across all three. this is used to prevent duplicate file upload.
### Motivation and Context
The `System.Linq.Async` package reference was removed from the core
libraries so that everyone using SK wasn't forced to use it as well.
This adds a helper `ToListAsync` method to enable async conversion of
`IAsyncEnumerable<T>` to `List<T>` object.
### Description
Was seeing the following error when using local project references, so
this is an effort to future proof any SK nuget updates
`'IAsyncEnumerable<MemoryQueryResult>' does not contain a definition for
'ToListAsync' and no accessible extension method 'ToListAsync' accepting
a first argument of type 'IAsyncEnumerable<MemoryQueryResult>' could be
found (are you missing a using directive or an assembly reference?) `
### Motivation and Context
CopilotChat webapi should use the standard health probe to aid in
systems integration.
### Description
- Removed probe controller
- Added basic ASP.NET health check on /healthz
- Updated all READMEs
- Add persist to Chroma on upsert or delete
- Fixes#1289
### Description
Added `self._client.persist()` in the `upsert_async` and
`delete_collection_async`.
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
The /invoke API in webapi is no longer being used by the webapp.
### Description
- Removed the SemanticKernelController from webapp
- Removed unused `invokeSkillWithMsalToken` and
`invokeSkillWithGraphToken` from `useConnectors.ts`
---------
Co-authored-by: Teresa Hoang <[email protected]>
### Motivation and Context
To work properly, multi-user chat needs one more CosmoDB container
(chatparticipants) and the web service needs to have web sockets enabled
in order to accomodate SignalR.
### Description
- Add creation of chatparticipants container
- Enabled web sockets on web service
### Motivation and Context
This PR sets up the support for HTTPS should developers choose to enable
it. Some customers have production sites using HTTPS and want the option
to have a local development site that behaves like HTTPS.
### Description
- Update WebApp env.example with required variables
- Updated README with instructions on how to generate locally signed
certificate and how to update copilot config files accordingly.
- Added `https://localhost:3000` to AllowedOrigins in webapi config
files
### Motivation and Context
PRs are blocked since the dotnet SDK analyzers introduced a back-compat
issue.
### Description
Brought in a global.json file to pin the SDK version until the next
servicing release of dotnet SDK.
### Motivation and Context
This PR fixes a regression introduced in #1075 and includes unit tests
to correctly validate that both string and integer property ids are
supported.
### Description
The changes are based on the code suggested by AwesomeYuer over in #794
- the JsonNumberHandling attribute behaved in an unexpected way and was
removed.
We're using very little of the library and can easily achieve the same
things without it. From the core libraries like abstractions, we
shouldn't force all consumers to take the additional 1MB dependency. And
from examples, we shouldn't force anyone copying/pasting to take the
additional dependency
Also removed unnecessary InternalsVisibleTo that were also interfering
with shared source in multiple assemblies.
Otherwise, someone with multiple package sources defined gets dozens of
warnings like the following when building the repo:
```
warning NU1507: There are 4 package sources defined in your configuration. When using central package management, please map your packa
ge sources with package source mapping (https://aka.ms/nuget-package-source-mapping) or specify a single package source. The following sources are defined: ...
```
### Motivation and Context
Bug: When you enter something into the message input box in Copilot Chat
but do not send it, and you switch to another chat session, the message
will appear in the other chat session's message input box.
Expected: You type a message in one of the chat sessions, the message
should only appear in the input box of that chat session. And when you
switch to another session, the input box should become empty or show
whatever is already in there previously.
### Description
1. Remove the feature that enables users to go back to the previous
message by pressing the up arrow key.
2. Fix the bug described above to enable the expected behavior.
### Motivation and Context
When using OpenAI in the GitHub Repo Q&A sample, an error occurs when a
valid chat completions model is chosen from the dropdown list.
Furthermore, selecting a text completions model passes validation but
causes a bad request error when saving.
### Description
Fixed issue by updating the model validation function to test model
selection against the chat completions endpoint rather than the text
completions endpoint.
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
DuckDB is becoming a popular option for in memory OLAP storage. This pr
contributes a MemoryStorage implementation on DuckDB using both tile and
in memory setup.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
DuckDB memory storage implementation
In the `skprompt.txt` file, the skill ` _GLOBAL_FUNCTIONS_.SendEmail`
has one input as `recipient` while the plan shows the input as
`email_address`.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
