Skip to content

Commit e818e68

Browse files
committed
HADOOP-13327 Josh's comments
Change-Id: I95e6b63eda051afae22a9d48c35b5e69e0464852
1 parent 9f4fb5c commit e818e68

2 files changed

Lines changed: 8 additions & 5 deletions

File tree

hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -636,7 +636,7 @@ The behavior of the returned stream is covered in [Output](outputstream.html).
636636
* S3A, Swift and potentially other Object Stores do not currently change the `FS` state
637637
until the output stream `close()` operation is completed.
638638
This is a significant difference between the behavior of object stores
639-
and that of filesystems, as it allows >1 client to create a file with `overwrite==false`,
639+
and that of filesystems, as it allows >1 client to create a file with `overwrite=false`,
640640
and potentially confuse file/directory logic. In particular, using `create()` to acquire
641641
an exclusive lock on a file (whoever creates the file without an error is considered
642642
the holder of the lock) may not not a safe algorithm to use when working with object stores.

hadoop-common-project/hadoop-common/src/site/markdown/filesystem/outputstream.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -884,7 +884,7 @@ This has been difficult to deal with within the Hadoop S3A code itself
884884
(HADOOP-16490, HADOOP-16635) -and if applications make their own probes for files
885885
before creating them, the problem will intermittently surface.
886886
887-
1. If you look for an object on S3 and it is not there - The 404 MAY Be returned even
887+
1. If you look for an object on S3 and it is not there - The 404 MAY be returned even
888888
after the object has been created.
889889
1. FS operations triggering such a probe include: `getFileStatus()`, `exists()`, `open()`
890890
and others.
@@ -894,7 +894,7 @@ create files with this option except when some form of exclusivity is needed on
894894
creation -in which case, be aware, that with the non-atomic probe+create sequence which
895895
some object store connectors implement, the semantics of the creation are not sufficient
896896
to allow the filesystem to be used as an implicit coordination mechanism between processes.
897-
``
897+
898898
## <a name="implementors"></a> Implementors notes.
899899
900900
### `StreamCapabilities`
@@ -914,9 +914,12 @@ on to the distributed FS, it SHOULD declare that it supports them.
914914
### Metadata updates
915915
916916
Implementors MAY NOT update a file's metadata (length, date, ...) after
917-
every `hsync()` call. HDFS doesn't -so there is no need to feel too guilty.
917+
every `hsync()` call. HDFS doesn't, except when the written data crosses
918+
a block boundary.
918919
919920
### Does `close()` sync data?
920921
921-
By default, HDFS does not sync data to disk when a stream is closed.
922+
By default, HDFS does not sync data to disk when a stream is closed; it will
923+
be asynchronously saved to disk.
924+
922925
This does not mean that users do not expect it.

0 commit comments

Comments
 (0)