[HDFS-8178] QJM doesn't move aside stale inprogress edits files - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 3.3.0, 3.2.1, 3.1.3
Component/s: qjm
Labels:
- BB2015-05-TBR

Description

When a QJM crashes, the in-progress edit log file at that time remains in the file system. When the node comes back, it will accept new edit logs and those stale in-progress files are never cleaned up. QJM treats them as regular in-progress edit log files and tries to finalize them, which potentially causes high memory usage. This JIRA aims to move aside those stale edit log files to avoid this scenario.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HDFS-8178.branch-3.1.patch
30/Aug/19 15:18
18 kB
István Fajth
HDFS-8178.branch-3.2.patch
30/Aug/19 00:03
19 kB
Wei-Chiu Chuang
HDFS-8178.008.patch
28/Aug/19 20:29
19 kB
István Fajth
HDFS-8178.008.merged
28/Aug/19 20:29
37 kB
István Fajth
HDFS-8178.008.addendum
28/Aug/19 20:29
26 kB
István Fajth
HDFS-8178.007.patch
23/Aug/19 12:03
16 kB
István Fajth
HDFS-8178.006.patch
23/Aug/19 10:09
16 kB
István Fajth
HDFS-8178.005.patch
22/Aug/19 17:42
16 kB
István Fajth
HDFS-8178.004.patch
22/Aug/19 16:57
14 kB
István Fajth
HDFS-8178.003.patch
30/Apr/15 22:02
4 kB
Zhe Zhang
HDFS-8178.002.patch
30/Apr/15 22:01
3 kB
Zhe Zhang
HDFS-8178.000.patch
22/Apr/15 23:10
4 kB
Zhe Zhang

Issue Links

relates to

HDFS-5919 FileJournalManager doesn't purge empty and corrupt inprogress edits files

Closed

requires

HDFS-8303 QJM should purge old logs in the current directory through FJM

Patch Available

Activity

People

Assignee:: István Fajth

Reporter:: Zhe Zhang

Votes:: 0 Vote for this issue

Watchers:: 9 Start watching this issue

Dates

Created:: 17/Apr/15 18:55

Updated:: 02/Oct/19 17:16

Resolved:: 30/Aug/19 20:15