Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2755

dfs fsck extremely slow, dfs ls times out

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.16.0
    • 0.16.0
    • None
    • None
    • Jan 28 nightly build

      With patches 2095, 2119, and 2723

    Description

      I upgraded a Hadoop installation to the Jan 28 nightly build.
      DFS contains 2.4+ M files.
      Upgrade finished but not finalized.

      Before finalizing I wanted to run fsck on the DFS. It hardly progressed after 6 hours (not finished yet). With the '-files' option turned on, it lists about 300 entries in 10 minutes.

      And when I tried to list a subdirectory with 100,000 files, it repeatedly (about 20 attempts) timed out.
      Changing timeout from 1 to 10 minutes did not help.

      Attachments

        1. 2755_20080131b.patch
          5 kB
          Tsz-wo Sze
        2. 2755_20080131.patch
          5 kB
          Tsz-wo Sze

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            szetszwo Tsz-wo Sze
            ckunz Christian Kunz
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment