Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2755

dfs fsck extremely slow, dfs ls times out

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.16.0
    • 0.16.0
    • None
    • None
    • Jan 28 nightly build

      With patches 2095, 2119, and 2723

    Description

      I upgraded a Hadoop installation to the Jan 28 nightly build.
      DFS contains 2.4+ M files.
      Upgrade finished but not finalized.

      Before finalizing I wanted to run fsck on the DFS. It hardly progressed after 6 hours (not finished yet). With the '-files' option turned on, it lists about 300 entries in 10 minutes.

      And when I tried to list a subdirectory with 100,000 files, it repeatedly (about 20 attempts) timed out.
      Changing timeout from 1 to 10 minutes did not help.

      Attachments

        1. 2755_20080131b.patch
          5 kB
          Tsz-wo Sze
        2. 2755_20080131.patch
          5 kB
          Tsz-wo Sze

        Issue Links

          Activity

            People

              szetszwo Tsz-wo Sze
              ckunz Christian Kunz
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: