[Cosmos][VectorSearch] Non Streaming Order By Query (#40096)

* Initial changes

* Initial changes

* Increment versions for core releases (#40003)

Increment package versions for core releases

* Ensure ServiceBus session idle timeout fall back to retry-options::try-timeout (#39994)

* Added Alpha3 Java Media Streaming Events (#40002)

* Added Alpha3 Java Media Streaming Events

* updating readme to add the media streaming events to remove model

---------

Co-authored-by: Vinothini Dharmaraj <[email protected]>

* Update version of github-event-processor to 1.0.0-dev.20240502.2 (#40012)

Co-authored-by: James Suplizio <[email protected]>

* Prepare May 2024 Identity Release (#40006)

* Prepare Identity Broker May 2024 Release (#40014)

* Increment package versions for identity releases (#40015)

* [JobRouter] SDK Review updates (#40011)

* SDK Review updates

* Update auto-generated models

* Add customization

* Fix customization

* Update package

* Update tests

* Linting

* FixFaultInjectionRuleFailedToApplyPerPartitionInGatewayMode (#40005)

* fix fault injection rule failed to apply per partition in gateway mode

---------

Co-authored-by: annie-mac <[email protected]>

* azure-cosmos-test_1.0.0.beta.7Release (#40021)

* release azure-cosmos-test 1.0.0.beta.7
---------

Co-authored-by: annie-mac <[email protected]>

* Fixed existsById API in ReactiveCosmosTemplate (#40022)

* Fixed existsById API in ReactiveCosmosTemplate

* Added changelog

* Initial changes

* Initial changes

* Skip Recorded test and delete Event record until test proxy to work with Event recordings (#40029)

Co-authored-by: Min Woo Lee 🧊 <[email protected]>

* Fix invalid CODEOWNERS (#40032)

* Initial changes

* ServiceBus: fix session tracing (#39962)

* remove additional matrix

* Fix session processing and disposition instrumentation

* return matrix config

* review suggestions

* [Automation] Generate SDK based on TypeSpec 0.15.15 (#40048)

* [CODEOWNERS] Updates for org changes (#40049)

* [CODEOWNERS] Updates for org changes

The focus of these changes is to remove an individual who no longer is responsible for the products which their GH account is associated to.

* Move from using the docker image to java2docfx for docs validation (#39744)

* Move from using the docker image to java2docfx for docs validation

* Temporarily turn on docs processing for template libraries for testing

* Actually install the rex validation tool

* Fix the if not Test-Path statement

* Update java2docfx version and add a couple of diagnostics output lines

* Add missing close paren

* Ensure that Sort-Object always returns an array even if there's only one item

* add another piece of diagnostics output

* trying one more thing

* remove some diag, add other

* Remove the additional diagnostics, add permanent output message

* Invoke java -jar on java2docfx to show the help command to ensure the install is okay

* fiddling with the java -jar command

* Set the working directory to the java2docfx directory before executing the mvn dependency download

* Actually create the directory before trying to set location...oops

* Update rex validation to verify MAVEN_HOME is set

* Updates for Java PR 39875 which had changes from this PR that were more immediate

* Update java2docfx version

* remove check for MAVEN_HOME which was only for testing

* Update the version of java2docfx to test a fix

* Update version of java2docfx to 1.0.4

* revert template's ci.yml changes that were only necessary to test java2docfx

* owners (#39686)

* Use ClientLogger in testing output (#40010)

Use ClientLogger in testing output

* Fix null pointer exception and context usage (#40053)

* Rename AML to AzureMachineLearning (#40056)

* Fixed the Key Vault `test-resources.json` file to properly configure a deployment script for certificate creation. (#40037)

* Close response body in bearer policy (#40052)

* Running Prepare-Release for azure-messaging-servicebus 7.17.0 (#40058)

* mgmt, TypeSpec code generation pipeline (#39963)

* typespec generation pipeline

echo command

PR_TITLE

* generation typespec

Update generation.yml for Azure Pipelines

Update generation.yml for Azure Pipelines

Update generation.yml for Azure Pipelines

* remove typespec pipeline file

* fix pr title

* address comments

* Add codeowner linter owners (#39997)

* Update to ESRP task version that supports federated auth (#40059)

* Increment package versions for cosmos releases (#40031)

* Update azure-sdk-build-tools Repository Resource Refs in Yaml files (#39627)

* Add reduced embeddings sample to azure-search-documents (#40069)

* Add reduced embeddings sample

* Fix cspell

* Fix link

* Search May Preview Regen Updates (#40057)

* Search May Preview Regeneration
- Still need to add varargs convenience

* Removing ovveride statements from `setFields` for `VectorizableImageUrlQuery` and `VectorizableImageBinaryQuery`

* Removing ovveride statements from `setFields` for `VectorizableImageUrlQuery` and `VectorizableImageBinaryQuery`

* adding varargs

* Additional adjustments to FieldBuilder and Search Index Customizations

* Updating cspell.json

* Adjust `SearchScoreThreshold` customization
Re-enable code generation in CI

* Updates:

- Updated Cspell
- Rename `maxStoragePerIndex` property to `maxStoragePerIndexInBytes` in SearchServiceLimits
- Set `hybridSearch` property to be type `HybridSearch` in SearchRequest
- Add `hybridSearch` to SearchOptions and `SearchAsyncClient.createSearchRequest()`

* Adding Support and testing byte[] and List<byte> within field builder

* Fix linting

---------

Co-authored-by: alzimmermsft <[email protected]>

* Preparing Search May 2024 Beta Release (#40071)

* Preparing Search May 2024 Beta Release

* Preparing Search May 2024 Beta Release

* Resolving comments

* Fixing build issues

* eng, update autorest.java, improve error output in sdk automation (#40073)

* improve error output

* autorest.java 4.1.29

* Merge to main after spring cloud azure 4.18.0 released (#40075)

* Prepare for Spring Cloud Azure 4.18.0 release (#40063)

* update version client

* update version/changelog/readme

* update changelog

* Increment versions for spring releases (#40074)

* Increment package versions for spring releases

* Update version_client.txt

* Update pom.xml

---------

Co-authored-by: Muyao Feng <[email protected]>

---------

Co-authored-by: Azure SDK Bot <[email protected]>

* Miscellaneous Core performance improvements (#39552)

Miscellaneous Core performance improvements

* Increment package versions for search releases (#40072)

* Update io.fabric8:kubernetes-client (#40086)

5.12.3 -> 6.12.1

* Increment package versions for servicebus releases (#40094)

* Emit stable auto-instrumented otel metrics (#39960)

* Update otel metrics logic

* add runtime metrics

* adding a few metrics I forgot

* small correction

* Update

* Fix

* Update

* Delete pre-stable metrics

---------

Co-authored-by: Harsimar Kaur (from Dev Box) <[email protected]>

---------

Co-authored-by: Azure SDK Bot <[email protected]>
Co-authored-by: Anu Thomas Chandy <[email protected]>
Co-authored-by: v-durgeshs <[email protected]>
Co-authored-by: Vinothini Dharmaraj <[email protected]>
Co-authored-by: James Suplizio <[email protected]>
Co-authored-by: Bill Wert <[email protected]>
Co-authored-by: williamzhao87 <[email protected]>
Co-authored-by: Annie Liang <[email protected]>
Co-authored-by: annie-mac <[email protected]>
Co-authored-by: Kushagra Thapar <[email protected]>
Co-authored-by: minwoolee-msft <[email protected]>
Co-authored-by: Min Woo Lee 🧊 <[email protected]>
Co-authored-by: Alan Zimmer <[email protected]>
Co-authored-by: Liudmila Molkova <[email protected]>
Co-authored-by: Jesse Squire <[email protected]>
Co-authored-by: Harsimar Kaur <[email protected]>
Co-authored-by: vcolin7 <[email protected]>
Co-authored-by: Xiaofei Cao <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: Patrick Hallisey <[email protected]>
Co-authored-by: Jair Myree <[email protected]>
Co-authored-by: Weidong Xu <[email protected]>
Co-authored-by: Muyao Feng <[email protected]>
Co-authored-by: Helen <[email protected]>
Co-authored-by: Harsimar Kaur (from Dev Box) <[email protected]>

Author: 24 people

.github/CODEOWNERS

@@ -396,10 +396,10 @@
 # ServiceOwners:                                     @raedJarrar @jifems
 
 # PRLabel: %DevCenter
-/sdk/devcenter/                                      @sebrenna @mharlan
+/sdk/devcenter/                                      @sebrenna
 
 # ServiceLabel: %DevCenter
-# ServiceOwners:                                     @sebrenna @mharlan
+# ServiceOwners:                                     @sebrenna
 
 # ServiceLabel: %Device Provisioning Service
 # ServiceOwners:                                     @nberdy
@@ -430,7 +430,7 @@
 
 # ServiceLabel: %Event Grid
 # AzureSdkOwners:                                   @billwert
-# ServiceOwners:                                    @jfggdl
+# ServiceOwners:                                    @Kishp01 @ahamad-MS
 
 # PRLabel: %Event Hubs
 /sdk/eventhubs/                                      @conniey @anuchandy @lmolkova
@@ -558,7 +558,7 @@
 # ServiceOwners:                                     @ambhatna @savjani
 
 # PRLabel:  %OpenTelemetry
-/sdk/monitor/azure-monitor-opentelemetry-exporter/   @trask @ramthi @heyams @jeanbisutti
+/sdk/monitor/azure-monitor-opentelemetry-exporter/   @trask @ramthi @heyams @jeanbisutti @harsimar
 
 # ServiceLabel: %Network
 # ServiceOwners:                                     @aznetsuppgithub
@@ -647,9 +647,6 @@
 # ServiceLabel: %Redis Cache
 # ServiceOwners:                                     @yegu-ms
 
-# ServiceLabel: %Relay
-# ServiceOwners:                                     @jfggdl
-
 # PRLabel: %Remote Rendering
 /sdk/remoterendering/                                @MichaelZp0 @ChristopherManthei
 
@@ -796,13 +793,13 @@
 /sdk/spring-experimental/                            @chenrujun @netyyyy @saragluna @moarychan
 
 # PRLabel:  %Monitor - Spring
-/sdk/spring/spring-cloud-azure-starter-monitor       @jeanbisutti @trask @ramthi @heyams
+/sdk/spring/spring-cloud-azure-starter-monitor       @jeanbisutti @trask @ramthi @heyams @harsimar
 
 # PRLabel:  %Monitor - Spring
-/sdk/spring/spring-cloud-azure-starter-monitor-test  @jeanbisutti @trask @ramthi @heyams
+/sdk/spring/spring-cloud-azure-starter-monitor-test  @jeanbisutti @trask @ramthi @heyams @harsimar
 
 # ServiceLabel:  %Monitor - Spring
-# AzureSdkOwners:                                    @jeanbisutti @trask @ramthi @heyams
+# AzureSdkOwners:                                    @jeanbisutti @trask @ramthi @heyams @harsimar
 
 # PRLabel: %azure-spring
 /sdk/spring/spring-cloud-azure-appconfiguration-config*/ @mrm9084 @chenrujun @netyyyy @saragluna @moarychan
@@ -853,3 +850,4 @@
 
 # Add owners for notifications for specific pipelines
 /eng/pipelines/aggregate-reports.yml                 @joshfree @jonathangiles
+/eng/common/pipelines/codeowners-linter.yml          @alzimmermsft @srnagar @lmolkova

.github/CODEOWNERS_baseline_errors.txt

@@ -267,3 +267,8 @@ ccmixpdevs is an invalid user. Ensure the user exists, is public member of Azure
 ccmbpxpcrew is an invalid user. Ensure the user exists, is public member of Azure and has write permissions.
 TiagoCrewGitHubIssues is an invalid user. Ensure the user exists, is public member of Azure and has write permissions.
 ccmshowbackdevs is an invalid user. Ensure the user exists, is public member of Azure and has write permissions.
+sagivf is not a public member of Azure.
+Aviv-Yaniv is not a public member of Azure.
+tmahmood-microsoft is not a public member of Azure.
+Kishp01 is not a public member of Azure.
+ahamad-MS is an invalid user. Ensure the user exists, is public member of Azure and has write permissions.

eng/emitter-package-lock.json

@@ -1,16 +1,16 @@
 {
   "main": "dist/src/index.js",
   "dependencies": {
-    "@azure-tools/typespec-java": "0.15.14"
+    "@azure-tools/typespec-java": "0.15.15"
   },
   "devDependencies": {
-    "@typespec/versioning": "0.55.0",
-    "@azure-tools/typespec-client-generator-core": "0.41.6",
-    "@typespec/http": "0.55.0",
-    "@typespec/openapi": "0.55.0",
+    "@azure-tools/typespec-client-generator-core": "0.41.8",
     "@typespec/rest": "0.55.0",
     "@typespec/compiler": "0.55.0",
+    "@typespec/openapi": "0.55.0",
     "@azure-tools/typespec-azure-core": "0.41.0",
+    "@typespec/versioning": "0.55.0",
+    "@typespec/http": "0.55.0",
     "@azure-tools/typespec-azure-resource-manager": "0.41.0",
     "@azure-tools/typespec-autorest": "0.41.1"
   }

eng/emitter-package.json

@@ -9,7 +9,7 @@
 from parameters import *
 from generate import update_parameters
 from generate_utils import get_version
-from generate import compile_package
+from generate_utils import compile_arm_package
 from generate import compare_with_maven_package
 
 os.chdir(pwd)
@@ -31,7 +31,7 @@ def main():
     update_parameters(args.get('suffix'))
 
     if args.get('compile'):
-        compile_package(sdk_root, service)
+        compile_arm_package(sdk_root, service)
 
     versions = get_version(sdk_root, service).split(';')
     stable_version = versions[1]

eng/mgmt/automation/changelog.py

@@ -24,7 +24,7 @@
 )
 from generate_utils import (
     compare_with_maven_package,
-    compile_package,
+    compile_arm_package,
     generate,
     get_and_update_service_from_api_specs,
     get_suffix_from_api_specs,
@@ -187,7 +187,7 @@ def sdk_automation_autorest(config: dict) -> List[dict]:
                 tag=tag,
             )
             if succeeded:
-                compile_package(sdk_root, module)
+                compile_arm_package(sdk_root, module)
 
             packages.append({
                 'packageName':
@@ -256,7 +256,7 @@ def sdk_automation_typespec_project(tsp_project: str, config: dict) -> dict:
             update_root_pom(sdk_root, service)
 
         # compile
-        succeeded = compile_package(sdk_root, module)
+        succeeded = compile_arm_package(sdk_root, module)
 
     # output
     if sdk_folder and module and service:
@@ -359,7 +359,7 @@ def main():
         )
 
     if succeeded:
-        succeeded = compile_package(sdk_root, module)
+        succeeded = compile_arm_package(sdk_root, module)
         if succeeded:
             compare_with_maven_package(sdk_root, service, stable_version,
                                        current_version, module)

eng/mgmt/automation/generate.py

@@ -335,7 +335,11 @@ def compile_package(sdk_root: str, group_id: str, module: str) -> bool:
         sdk_root, group_id, module)
     logging.info(command)
     if os.system(command) != 0:
-        logging.error('[COMPILE] Maven build fail')
+        error_message = ('[COMPILE] Maven build fail.\n'
+                         'One reason of the compilation failure is that the existing code customization in SDK repository being incompatible with the class generated from updated TypeSpec source. In such case, you can ignore the failure, and fix the customization in SDK repository.\n'
+                         'You can inquire in "Language - Java" Teams channel. Please include the link of this Pull Request in the query.')
+        logging.error(error_message)
+        print(error_message, file=sys.stderr)
         return False
     return True
 

eng/mgmt/automation/generate_data.py

@@ -95,7 +95,7 @@ def generate(
     return True
 
 
-def compile_package(sdk_root, module) -> bool:
+def compile_arm_package(sdk_root: str, module: str) -> bool:
     if os.system(
             'mvn --no-transfer-progress clean verify -f {0}/pom.xml -Dmaven.javadoc.skip -Dgpg.skip -DskipTestCompile -Djacoco.skip -Drevapi.skip -pl {1}:{2} -am'.format(
                 sdk_root, GROUP_ID, module)) != 0:

eng/mgmt/automation/generate_utils.py

@@ -12,7 +12,9 @@ variables:
 - name: MAVEN_OPTS
   value: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
 - name: NodeVersion
-  value: '16.x'
+  value: '18.x'
+- name: fromTypeSpec
+  value: $[ne(variables['TSP_CONFIG'], '')]
 
 steps:
 - bash: |
@@ -29,6 +31,12 @@ steps:
 - bash: |
     npm install -g autorest
   displayName: 'Install autorest'
+  condition: eq(variables.fromTypeSpec, false)
+
+- bash: |
+    npm install -g @azure-tools/typespec-client-generator-cli
+  displayName: 'Install tsp-client'
+  condition: eq(variables.fromTypeSpec, true)
 
 # - template: /eng/common/testproxy/test-proxy-tool.yml
 #   parameters:
@@ -38,13 +46,27 @@ steps:
 - bash: |
     export PATH=$JAVA_HOME_11_X64/bin:$PATH
     java -version
+    set -x
+    ./eng/mgmt/automation/generate.py --tsp-config "$(TSP_CONFIG)" --version "$(VERSION)" --auto-commit-external-change --user-name "azure-sdk" --user-email "[email protected]"
+  displayName: Generation from TypeSpec
+  condition: eq(variables.fromTypeSpec, true)
+
+- bash: |
+    export PATH=$JAVA_HOME_11_X64/bin:$PATH
+    java -version
+    set -x
     ./eng/mgmt/automation/generate.py --readme "$(README)" --tag "$(TAG)" --autorest-options="$(AUTOREST_OPTIONS)" --service "$(SERVICE)" --version "$(VERSION)" --suffix "$(SUFFIX)" --auto-commit-external-change --user-name "azure-sdk" --user-email "[email protected]"
-  displayName: Generation
+  displayName: Generation from Swagger
+  condition: eq(variables.fromTypeSpec, false)
 
 - template: /eng/common/pipelines/templates/steps/create-pull-request.yml
   parameters:
     PRBranchName: fluent-lite-generation-$(Build.BuildId)
-    CommitMsg: '[Automation] Generate Fluent Lite from $(README)#$(TAG)'
-    PRTitle: '[Automation] Generate Fluent Lite from $(README)#$(TAG)'
+    ${{ if variables.fromTypeSpec }}:
+      CommitMsg: '[Automation] Generate Fluent Lite from TypeSpec $(README)'
+      PRTitle: '[Automation] Generate Fluent Lite from TypeSpec $(README)'
+    ${{ else }}:
+      CommitMsg: '[Automation] Generate Fluent Lite from Swagger $(README)#$(TAG)'
+      PRTitle: '[Automation] Generate Fluent Lite from Swagger $(README)#$(TAG)'
     PRLabels: 'Mgmt - Track 2'
     OpenAsDraft: '$(DRAFT_PULL_REQUEST)'

eng/mgmt/automation/generation.yml

@@ -12,7 +12,7 @@ variables:
 - name: MAVEN_OPTS
   value: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
 - name: NodeVersion
-  value: '16.x'
+  value: '18.x'
 
 steps:
 - bash: |