[HDFS-11229] HDFS-11056 failed to close meta file

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.7.4, 3.0.0-alpha2
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
• Component/s: datanode
• Target Version/s: 2.7.4, 3.0.0-alpha2

Release Note

The fix for HDFS-11056 reads meta file to load last partial chunk checksum when a block is converted from finalized/temporary to rbw. However, it did not close the file explicitly, which may cause number of open files reaching system limit. This jira fixes it by closing the file explicitly after the meta file is read.

• Assignee: Wei-Chiu Chuang
• Reporter: Wei-Chiu Chuang
• Created: 09/Dec/16 20:05
• Updated: 19/Dec/16 15:20
• Resolved: 10/Dec/16 17:35

Description

The following code failed to close the file after it is read.

FsVolumeImpl#loadLastPartialChunkChecksum

    RandomAccessFile raf = new RandomAccessFile(metaFile, "r");
    raf.seek(offsetInChecksum);
    raf.read(lastChecksum, 0, checksumSize);
    return lastChecksum;

This must be fixed because every append operation uses this piece of code. Without an explicit close, open files can reach system limit before RandomAccessFile objects are garbage collected.

Issue Links

• is broken by: HDFS-11056 Concurrent append and read operations lead to checksum error
• is depended upon by: HDFS-11160 VolumeScanner reports write-in-progress replicas as corrupt incorrectly
• is related to: HDFS-11160 VolumeScanner reports write-in-progress replicas as corrupt incorrectly