[MAPREDUCE-865] harchive: Reduce the number of open calls to _index and _masterindex

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Component/s: harchive
• Assignee: Koji Noguchi
• Reporter: Koji Noguchi
• Created: 13/Aug/09 23:43
• Updated: 22/Jul/13 14:56
• Resolved: 22/Jul/13 14:56

Description

When I have har file with 1000 files in it,
% hadoop dfs -lsr har:///user/knoguchi/myhar.har/
would open/read/close the _index/_masterindex files 1000 times.

This makes the client slow and add some load to the namenode as well.
Any ways to reduce this number?

Issue Links

• duplicates: MAPREDUCE-2459 Cache HAR filesystem metadata