Skip to content

Retry-reason in x-ms-client-request-id #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
saxenapranav merged 2 commits into from
Jun 19, 2023
Merged

Retry-reason in x-ms-client-request-id #64

saxenapranav merged 2 commits into from
Jun 19, 2023

Conversation

@saxenapranav
Copy link
Collaborator

@saxenapranav saxenapranav commented Jun 1, 2023 •
edited
Loading

taking code from:
apache#5299: apache@358bf80

apache#5437: apache@759ddeb.

ex of clientRequestId: cid=:f0918642-ef5b-4687-8f31-f6dc90306fbe:db424ea7-6849-4dd4-a4e0-0dad8c621fe0:bff5b9a81814::CF:2_RT:B

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:365 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 3, Skipped: 226
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 53

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 182
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestExponentialRetryPolicy.testCreateMultipleAccountThrottling:165 [iff singleton is not enabled, for different accounts instances should return different value]
Expecting:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@5c04970e
not to be equal to:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@5c04970e

[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:191 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=141).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 2, Errors: 1, Skipped: 274
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

NonHNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestExponentialRetryPolicy.testCreateMultipleAccountThrottling:165 [iff singleton is not enabled, for different accounts instances should return different value]
Expecting:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@7e2f14be
not to be equal to:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@7e2f14be

[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:191 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=137).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemFlush.testTracingHeaderForAppendBlob:319 » IO AppendBl...
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AbfsRestOperation O...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 2, Errors: 4, Skipped: 280
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 57

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=120).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:365 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 3, Skipped: 226
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 53

Time taken: 48 mins 12 secs.

@saxenapranav
HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried…
4ad9424 
… API call. (apache#5299)

Contributed by Pranav Saxena

(cherry picked from commit 358bf80)
@saxenapranav
HADOOP-18647. x-ms-client-request-id to identify the retry of an API. (…
89a7096 
…apache#5437)

The x-ms-client-request-id now includes a field to indicate a call is a retry of a previous
operation

Contributed by Pranav Saxena

(cherry picked from commit 759ddeb)
@saxenapranav saxenapranav marked this pull request as ready for review June 1, 2023 09:23
@saxenapranav
Copy link
Collaborator Author

saxenapranav commented Jun 1, 2023

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:360 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:181->fuzzyValidate:64 The actual value 9 is not within the expected range: [5.60, 8.40].
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=90).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 2, Skipped: 275
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

NonHNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=281).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 1, Skipped: 275
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=105).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

Time taken: 47 mins 16 secs.

anujmodi2021
anujmodi2021 approved these changes Jun 9, 2023
View reviewed changes
Copy link
Collaborator

@anujmodi2021 anujmodi2021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given both the commits are Cherry-picked.
This LGTM

@saxenapranav saxenapranav merged commit 4a47e8b into ABFSDriver:ABFS_3.3.2_dev Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anmolanmol1234 anmolanmol1234 anmolanmol1234 approved these changes

@anujmodi2021 anujmodi2021 anujmodi2021 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@saxenapranav @anmolanmol1234 @anujmodi2021