HADOOP-13655 document object store use with fs shell and distcp #131
Conversation
There was a problem hiding this comment.
hadoop fs -copyFromLocal -d -f /datasets/devices.orc s3a://bucket/datasets/" is redundant, right?
The symbol "
There was a problem hiding this comment.
hadoop fs -put -d -f - wasb: should be hadoop fs -put -d -f wasb:
There was a problem hiding this comment.
actually no! I couldn't get the put command to read off stdin according to the (Existing) docs, so I looked in the source code and saw that you needed a special "-" as the source, otherwise the s3a url :// was being treated as a source and the command failing as the destination was missing. It does make sense —and is consistent with other tools, just not documented completely.
I'd mentioned this in the "put" section, but I've expanded the comment here and added the - to the DNF form of the put command.
steveloughran
changed the title
liuml07
left a comment
liuml07
left a comment
There was a problem hiding this comment.
+1
This is excellent doc; we should get this in ASAP.
This is for branch-2? How about trunk?
There was a problem hiding this comment.
<a name="ObjectStores" /> is accidently here I guess?
There was a problem hiding this comment.
no, it's to have a link which can be referred to later
There was a problem hiding this comment.
The -diff/-rdiff option is not supported
Yes there is an rdiff options that is just added.
There was a problem hiding this comment.
it is needed to indent the paragraph and code sample under the bullet point.
Change-Id: I0f50d0a7823a86a9e0b1fdfc70a69fcebd1d22ce
steveloughran
force-pushed
the
s3/HADOOP-13655-shell-docs
branch
from
0fea00c to
881366f
Compare
Join TTL was set too low (10 ms). `joinRetainsMatchedMessagesReverse` will fail if the execution time between line 176 and 186 is longer than that. Increased TTL to 1 min (will not affect test execution time). Author: Prateek Maheshwari <[email protected]> Reviewers: Navina Ramesh <[email protected]> Closes apache#131 from prateekm/join-test-fix
3 participants
Patch of filesystem shell & distcp docs to cover object stores. Also updated some references in filesystem/index.md which were out of date