Skip to content

Conversation

@mukund-thakur
Copy link
Contributor

@mukund-thakur mukund-thakur commented Oct 19, 2020

Re ran the failing test with s3 and gcs endpoints. All good.

@mukund-thakur
Copy link
Contributor Author

CC @steveloughran @sidseth

@bgaborg
Copy link

bgaborg commented Oct 21, 2020

change makes sense to me, we don't need to have this specific client created here.
+1

@steveloughran steveloughran merged commit 7f8ef76 into apache:trunk Oct 21, 2020
@steveloughran
Copy link
Contributor

oops, merged before yetus. Let's see what happens and whether I should revert. Yetus isn't going to run this test anyway

@steveloughran
Copy link
Contributor

having problems with this and a bucket in ireland where I haven't fixed the endpoint up

[ERROR] testCustomSignerAndInitializer(org.apache.hadoop.fs.s3a.auth.ITestCustomSigner)  Time elapsed: 2.368 s  <<< ERROR!
org.apache.hadoop.fs.s3a.AWSRedirectException: getFileStatus on s3a://stevel-ireland/test: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: eu-west-1. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: SJEZVWBWMKBDV37V; S3 Extended Request ID: UqxxO/6T1TrsbAE7rs3r/FrExYA8ealw8h+Zr+AxiM6ZxoFu6ZslTqUryhcsaT6XG5DrxtUccTg=; Proxy: null), S3 Extended Request ID: UqxxO/6T1TrsbAE7rs3r/FrExYA8ealw8h+Zr+AxiM6ZxoFu6ZslTqUryhcsaT6XG5DrxtUccTg=:301 Moved Permanently: The bucket is in this region: eu-west-1. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: SJEZVWBWMKBDV37V; S3 Extended Request ID: UqxxO/6T1TrsbAE7rs3r/FrExYA8ealw8h+Zr+AxiM6ZxoFu6ZslTqUryhcsaT6XG5DrxtUccTg=; Proxy: null)
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:236)
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:170)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3278)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3177)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerMkdirs(S3AFileSystem.java:3000)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:2971)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2387)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:361)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:203)
	at org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:77)
	at org.apache.hadoop.fs.s3a.auth.ITestCustomSigner.setup(ITestCustomSigner.java:67)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: eu-west-1. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: SJEZVWBWMKBDV37V; S3 Extended Request ID: UqxxO/6T1TrsbAE7rs3r/FrExYA8ealw8h+Zr+AxiM6ZxoFu6ZslTqUryhcsaT6XG5DrxtUccTg=; Proxy: null), S3 Extended Request ID: UqxxO/6T1TrsbAE7rs3r/FrExYA8ealw

This is in #2778 but I've just replicated it in trunk too.

steveloughran pushed a commit to steveloughran/hadoop that referenced this pull request Mar 24, 2021
asfgit pushed a commit that referenced this pull request Mar 25, 2021
…nts (#2395)

Contributed by Mukund Thakur

Change-Id: Ia5def405056691c349cf05530fd3172047d2058b
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
…nts (apache#2395)

Contributed by Mukund Thakur

Change-Id: Ib8dd09b69ce68d6cc8cb6c1cb09d755992483ee4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants