Skip to content

HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes #5240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
apurtell merged 2 commits into from
May 25, 2023
Merged

HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes #5240

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4cee309
HBASE-27843 If moveAndClose fails HFileArchiver should delete any inc…
apurtell May 15, 2023
110db1c
Address review feedback
apurtell May 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -569,8 +569,22 @@ private static boolean resolveAndArchiveFile(Path archiveDir, File currentFile,
+ " because it does not exist! Skipping and continuing on.", fnfe);
success = true;
} catch (IOException e) {
LOG.warn("Failed to archive " + currentFile + " on try #" + i, e);
success = false;
// When HFiles are placed on a filesystem other than HDFS a rename operation can be a
// non-atomic file copy operation. It can take a long time to copy a large hfile and if
// interrupted there may be a partially copied file present at the destination. We must
// remove the partially copied file, if any, or otherwise the archive operation will fail
// indefinitely from this point.
LOG.warn("Failed to archive " + currentFile + " on try #" + i, e);
try {
fs.delete(archiveFile, false);
} catch (FileNotFoundException fnfe) {
// This case is fine.
} catch (IOException ee) {
// Complain about other IO exceptions
LOG.warn("Failed to clean up from failure to archive " + currentFile + " on try #" + i,
ee);
}
}
}

Expand Down