.Net: Added dimensions property to OpenAI embedding generation services

[dmytrostruk]

Motivation and Context

Resolves: #6026

This PR contains changes to expose dimensions property which is supported by OpenAI and Azure .NET SDK:
https://platform.openai.com/docs/api-reference/embeddings/create#embeddings-create-dimensions

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[rogerbarreto]

Adding this parameter at the connector level seems to be an issue, we might consider having a ExecutionSettings for embeddings API...

Any change on this matter I would add as experimental if the plan is to soon move this to be a setting per Invocation.

@westey-m

[dmytrostruk]

Adding this parameter at the connector level seems to be an issue, we might consider having a ExecutionSettings for embeddings API...

@rogerbarreto I thought about that, but I also think that we can have both options supported. If you set vector dimensions once on connector level, you won't have to do that with each request. And I don't think that this parameter will change too much in runtime, because every time you change the dimension, your data may lose a context.

ExecutionSettings for embedding API will be supported, here is ADR for this:
#5277

Any change on this matter I would add as experimental if the plan is to soon move this to be a setting per Invocation.

All embedding functionality is marked as Experimental at the moment.

[rogerbarreto]

IF this is Experimental, suggest creating a new method marked as Experimental with the new dimension extra parameter to avoid the breaking change.

[dmytrostruk]

IF this is Experimental, suggest creating a new method marked as Experimental with the new dimension extra parameter to avoid the breaking change.

@rogerbarreto I thought about that as well :) But the thing is that there are 11 methods that were modified. In order to avoid breaking changes, I will need to add 11 more overload methods with new parameter. It's not a big problem, but since we agreed that at some point we will have to obsolete existing methods and replace them with Config classes, I'm not sure if we need to add new methods now in order to obsolete them later.

Besides, this is breaking change only from API compatibility point of view, but not for compiler, that's the reason I added this parameter as last one. And since these are static methods, it's not possible to mock them in tests, so it should not break mocking scenarios.