Skip to content

HDFS-16354. Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc. #3740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iwasakims merged 2 commits into from
Dec 7, 2021
Merged

HDFS-16354. Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc. #3740

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5526f16
HDFS-16354. Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc.
iwasakims Dec 1, 2021
10de5b8
added description about default values of request params.
iwasakims Dec 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ The HTTP REST API supports the complete [FileSystem](../../api/org/apache/hadoop
* [`GETALLSTORAGEPOLICY`](#Get_all_Storage_Policies) (see [FileSystem](../../api/org/apache/hadoop/fs/FileSystem.html).getAllStoragePolicies)
* [`GETSTORAGEPOLICY`](#Get_Storage_Policy) (see [FileSystem](../../api/org/apache/hadoop/fs/FileSystem.html).getStoragePolicy)
* [`GETSNAPSHOTDIFF`](#Get_Snapshot_Diff)
* [`GETSNAPSHOTDIFFLISTING`](#Get_Snapshot_Diff_Iteratively)
* [`GETSNAPSHOTTABLEDIRECTORYLIST`](#Get_Snapshottable_Directory_List)
* [`GETSNAPSHOTLIST`](#Get_Snapshot_List)
* [`GETFILEBLOCKLOCATIONS`](#Get_File_Block_Locations) (see [FileSystem](../../api/org/apache/hadoop/fs/FileSystem.html).getFileBlockLocations)
Expand Down Expand Up @@ -1604,6 +1605,26 @@ See also: [FileSystem](../../api/org/apache/hadoop/fs/FileSystem.html).renameSna

{"SnapshotDiffReport":{"diffList":[],"fromSnapshot":"s3","snapshotRoot":"/foo","toSnapshot":"s4"}}

### Get Snapshot Diff Iteratively

* Submit a HTTP GET request.

curl -X GET curl -i -X GET "http://localhost:9870/webhdfs/v1/foo?op=GETSNAPSHOTDIFFLISTING&oldsnapshotname=s4&snapshotname=s5"
Copy link
Contributor

@virajjasani virajjasani Dec 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to mentioned that in the absence of snapshotdiffstartpath and snapshotdiffindex, their default values used will be empty string and -1 respectively.

Copy link
Member Author

@iwasakims iwasakims Dec 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added description about default values of the parameters.


curl -i GET "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETSNAPSHOTDIFFLISTING
&oldsnapshotname=<SNAPSHOTNAME>&snapshotname=<SNAPSHOTNAME>&snapshotdiffstartpath=<STARTPATH>&snapshotdiffindex=<STARTINDEX>

The client receives a response with a
[`SnapshotDiffReportListing` JSON object](#SnapshotDiffReportListing_JSON_Schema).
The value of `lastPath` and `lastIndex` must be specified as
the value of `snapshotdiffstartpath` and `snapshotdiffindex` respectively on next iteration.

HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked

{"SnapshotDiffReportListing":{"createList":[],"deleteList":[],"isFromEarlier":true,"lastIndex":-1,"lastPath":"","modifyList":[]}}

### Get Snapshottable Directory List

* Submit a HTTP GET request.
Expand Down Expand Up @@ -2665,6 +2686,109 @@ var diffReportEntries =
}
```

### SnapshotDiffReportListing JSON Schema

```json
{
"name": "SnapshotDiffReportListing",
"type": "object",
"properties":
{
"SnapshotDiffReportListing":
{
"type" : "object",
"properties" :
{
"isFromEarlier":
{
"description" : "the diff is calculated from older to newer snapshot or not",
"type" : "boolean",
"required" : true
},
"lastIndex":
{
"description" : "the last index of listing iteration",
"type" : "integer",
"required" : true
},
"lastPath":
{
"description" : "String representation of the last path of the listing iteration",
"type" : "string",
"required" : true
},
"modifyList":
{
"description": "An array of DiffReportListingEntry",
"type" : "array",
"items" : diffReportListingEntries,
"required" : true
},
"createList":
{
"description": "An array of DiffReportListingEntry",
"type" : "array",
"items" : diffReportListingEntries,
"required" : true
},
"deleteList":
{
"description": "An array of DiffReportListingEntry",
"type" : "array",
"items" : diffReportListingEntries,
"required" : true
}
}
}
}
}
```

#### DiffReportListing Entries

JavaScript syntax is used to define `diffReportEntries` so that it can be referred in `SnapshotDiffReport` JSON schema.

```javascript
var diffReportListingEntries =
{
"type": "object",
"properties":
{
"dirId":
{
"description" : "inode id of the directory",
"type" : "integer",
"required" : true
},
"fileId":
{
"description" : "inode id of the file",
"type" : "integer",
"required" : true
},
"isRereference":
{
"description" : "this is reference or not",
"type" : "boolean",
"required" : true
},
"sourcePath":
{
"description" : "string representation of path where changes have happened",
"type" : "string",
"required" : true
},
"targetPath":
{
"description" : "string representation of target path of rename op",
"type" : "string",
"required" : false
}
}
}
```


### SnapshottableDirectoryList JSON Schema

```json
Expand Down