[HADOOP-2755] dfs fsck extremely slow, dfs ls times out

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.16.0

Environment

Jan 28 nightly build

With patches 2095, 2119, and 2723

• Assignee: Tsz-wo Sze
• Reporter: Christian Kunz
• Created: 31/Jan/08 17:15
• Updated: 08/Jul/09 16:42
• Resolved: 01/Feb/08 23:28

Description

I upgraded a Hadoop installation to the Jan 28 nightly build.
DFS contains 2.4+ M files.
Upgrade finished but not finalized.

Before finalizing I wanted to run fsck on the DFS. It hardly progressed after 6 hours (not finished yet). With the '-files' option turned on, it lists about 300 entries in 10 minutes.

And when I tried to list a subdirectory with 100,000 files, it repeatedly (about 20 attempts) timed out.
Changing timeout from 1 to 10 minutes did not help.

Issue Links

• is related to: HADOOP-2606 Namenode unstable when replicating 500k blocks at once