Skip to content

HDDS-1813. Fix false warning from ozones3 acceptance test. Contribute…#1107

Merged
xiaoyuyao merged 1 commit into
apache:trunkfrom
xiaoyuyao:HDDS-1813
Jul 16, 2019
Merged

HDDS-1813. Fix false warning from ozones3 acceptance test. Contribute…#1107
xiaoyuyao merged 1 commit into
apache:trunkfrom
xiaoyuyao:HDDS-1813

Conversation

@xiaoyuyao
Copy link
Copy Markdown
Contributor

…d by Xiaoyu Yao.

@xiaoyuyao xiaoyuyao requested a review from bharatviswa504 July 16, 2019 19:14
@xiaoyuyao xiaoyuyao self-assigned this Jul 16, 2019
Copy link
Copy Markdown
Contributor

@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM.
Me too seen this issue in acceptance run test results but never thought this is a space issue.
Thank You @xiaoyuyao for the fix.

@xiaoyuyao xiaoyuyao merged commit 71c21f0 into apache:trunk Jul 16, 2019
@xiaoyuyao xiaoyuyao deleted the HDDS-1813 branch July 16, 2019 20:48
@hadoop-yetus
Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 40 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 476 trunk passed
+1 compile 260 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 1517 branch has no errors when building and testing our client artifacts.
+1 javadoc 164 trunk passed
_ Patch Compile Tests _
+1 mvninstall 441 the patch passed
+1 compile 270 the patch passed
+1 javac 270 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 696 patch has no errors when building and testing our client artifacts.
+1 javadoc 160 the patch passed
_ Other Tests _
-1 unit 191 hadoop-hdds in the patch failed.
-1 unit 2005 hadoop-ozone in the patch failed.
+1 asflicense 58 The patch does not generate ASF License warnings.
5749
Reason Tests
Failed junit tests hadoop.ozone.container.ozoneimpl.TestOzoneContainer
hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
hadoop.ozone.client.rpc.TestFailureHandlingByClient
hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
hadoop.ozone.client.rpc.TestOzoneRpcClient
hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient
hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion
hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
hadoop.ozone.client.rpc.TestWatchForCommit
Subsystem Report/Notes
Docker Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1107/1/artifact/out/Dockerfile
GITHUB PR #1107
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient
uname Linux 77c7c4e3eec7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / c5e3ab5
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1107/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1107/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1107/1/testReport/
Max. process+thread count 5387 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1107/1/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
… only

During startup, the CSM cleans up the base directories for stores. Ideally, this cleanup should only be done for non-logged stores. In our current setup, if the applications configure both logged and non-logged store base directories to be the same, we delete the logged store directories as well which results in unnecessary bootstrap.

In this PR,
 * We modify the clean up behavior on startup to impact only non-logged stores
 * Update the configuration documentation to reflect potential implications of not configuring `job.logged.store.base.dir` and `job.non.logged.store.base.dir` for stateful applications.
 * Add a warning if both logged and non-logged store base directories are configured to use same path.

Author: mynameborat <[email protected]>

Reviewers: Hai Lu <[email protected]>

Closes apache#1107 from mynameborat/SAMZA-2274 and squashes the following commits:

7d47a90b [mynameborat] Fix the logged store condition check
35b1bb3d [mynameborat] Cleanup store directories on startup for non-logged store only
amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants