Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-786

Implement getContentSummary(..) in HftpFileSystem

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HftpFileSystem does not override getContentSummary(..). As a result, it uses FileSystem's default implementation, which computes content summary on the client side by calling listStatus(..) recursively. In contrast, DistributedFileSystem has overridden getContentSummary(..) and does the computation on the NameNode.

      As a result, running "fs -dus" on hftp is much slower than running it on hdfs.

      1. h786_20100223_0.20.patch
        12 kB
        Suresh Srinivas
      2. h786_20100106.patch
        12 kB
        Tsz Wo Nicholas Sze
      3. h786_20100106_0.20.patch
        12 kB
        Tsz Wo Nicholas Sze
      4. h786_20100104.patch
        12 kB
        Tsz Wo Nicholas Sze
      5. h786_20091224.patch
        12 kB
        Tsz Wo Nicholas Sze
      6. h786_20091223.patch
        11 kB
        Tsz Wo Nicholas Sze

        Activity

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development