Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-786

Implement getContentSummary(..) in HftpFileSystem

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HftpFileSystem does not override getContentSummary(..). As a result, it uses FileSystem's default implementation, which computes content summary on the client side by calling listStatus(..) recursively. In contrast, DistributedFileSystem has overridden getContentSummary(..) and does the computation on the NameNode.

      As a result, running "fs -dus" on hftp is much slower than running it on hdfs.

        Attachments

        1. h786_20100223_0.20.patch
          12 kB
          Suresh Srinivas
        2. h786_20100106.patch
          12 kB
          Tsz Wo Nicholas Sze
        3. h786_20100106_0.20.patch
          12 kB
          Tsz Wo Nicholas Sze
        4. h786_20100104.patch
          12 kB
          Tsz Wo Nicholas Sze
        5. h786_20091224.patch
          12 kB
          Tsz Wo Nicholas Sze
        6. h786_20091223.patch
          11 kB
          Tsz Wo Nicholas Sze

          Activity

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: