Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-786

Implement getContentSummary(..) in HftpFileSystem

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • None
    • None
    • Reviewed

    Description

      HftpFileSystem does not override getContentSummary(..). As a result, it uses FileSystem's default implementation, which computes content summary on the client side by calling listStatus(..) recursively. In contrast, DistributedFileSystem has overridden getContentSummary(..) and does the computation on the NameNode.

      As a result, running "fs -dus" on hftp is much slower than running it on hdfs.

      Attachments

        1. h786_20091223.patch
          11 kB
          Tsz-wo Sze
        2. h786_20091224.patch
          12 kB
          Tsz-wo Sze
        3. h786_20100104.patch
          12 kB
          Tsz-wo Sze
        4. h786_20100106.patch
          12 kB
          Tsz-wo Sze
        5. h786_20100106_0.20.patch
          12 kB
          Tsz-wo Sze
        6. h786_20100223_0.20.patch
          12 kB
          Suresh Srinivas

        Activity

          People

            szetszwo Tsz-wo Sze
            szetszwo Tsz-wo Sze
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: