[HDFS-2718] Optimize OP_ADD in edits loading

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0, 1.0.0, 2.0.0-alpha
• Fix Version/s: 0.22.1
• Component/s: namenode
• Target Version/s: 0.23.1, 0.22.1
• Hadoop Flags: Reviewed
• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Created: 22/Dec/11 19:35
• Updated: 10/Mar/15 04:35
• Resolved: 02/Feb/12 19:13

Description

During loading the edits journal FSEditLog.loadEditRecords() processes OP_ADD inefficiently. It first removes the existing INodeFile from the directory tree, then adds it back as a regular INodeFile, and then replaces it with INodeFileUnderConstruction if files is not closed. This slows down edits loading. OP_ADD should be done in one shot and retain previously existing data.

Sub-Tasks

• CreateEditLogs should generate a realistic edit log.: Resolved