[HDFS-4479] logSync() with the FSNamesystem lock held in commitBlockSynchronization

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0
• Hadoop Flags: Reviewed
• Assignee: Jing Zhao
• Reporter: Jing Zhao
• Created: 08/Feb/13 02:20
• Updated: 15/May/13 05:16
• Resolved: 24/Feb/13 21:56

Description

In FSNamesystem#commitBlockSynchronization of branch-1, logSync() may be called when the FSNamesystem lock is held. Similar to HDFS-4186, this may cause some performance issue.

The following issue was observed in a cluster that was running a Hive job and was writing to 100,000 temporary files (each task is writing to 1000s of files). When this job is killed, a large number of files are left open for write. Eventually when the lease for open files expires, lease recovery is started for all these files in a very short duration of time. This causes a large number of commitBlockSynchronization where logSync is performed with the FSNamesystem lock held. This overloads the namenode resulting in slowdown.

Since logSync is called right after the synchronization section, we can simply remove the logSync call.

Issue Links

• is related to: HDFS-4186 logSync() is called with the write lock held while releasing lease