[HDFS-8965] Harden edit log reading code against out of memory errors - ASF JIRA

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 2.0.0-alpha
Fix Version/s: 2.8.0, 3.0.0-alpha1
Component/s: None
Labels:
None

Target Version/s:

2.8.0

Description

We should harden the edit log reading code against out of memory errors. Now that each op has a length prefix and a checksum, we can validate the checksum before trying to load the Op data. This should avoid out of memory errors when trying to load garbage data as Op data.

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

HDFS-8965.001.patch
26/Aug/15 22:58
18 kB
Colin P. McCabe
HDFS-8965.002.patch
26/Aug/15 23:06
18 kB
Colin P. McCabe
HDFS-8965.003.patch
27/Aug/15 01:40
19 kB
Colin P. McCabe
HDFS-8965.004.patch
28/Aug/15 00:54
24 kB
Colin P. McCabe
HDFS-8965.005.patch
28/Aug/15 22:31
25 kB
Colin P. McCabe
HDFS-8965.006.patch
31/Aug/15 18:43
25 kB
Colin P. McCabe
HDFS-8965.007.patch
31/Aug/15 20:46
25 kB
Colin P. McCabe

Issue Links

relates to

HDFS-5995 TestFSEditLogLoader#testValidateEditLogWithCorruptBody gets OutOfMemoryError and dumps heap.

Resolved

Activity

People

Assignee:

Colin P. McCabe

Reporter:

Colin P. McCabe

Votes:

0 Vote for this issue

Watchers:

12 Start watching this issue

Dates

Created:

26/Aug/15 22:48

Updated:

30/Aug/16 01:28

Resolved:

01/Sep/15 01:07