Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6870

Add FileSystem#listLocatedStatus to list a directory's content together with each file's block locations

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This jira implements the new FileSystem API as proposed in HDFS-202. The new API aims to eliminate individual "getFileBlockLocations" calls to NN for each file in the input directory of a job. Instead, a file's block locations are returned together with FileStatus when listing a directory, thus improving getSplits performance.

        Attachments

        1. listFiles4.patch
          22 kB
          Hairong Kuang
        2. listFiles3.patch
          21 kB
          Hairong Kuang
        3. listFiles2.patch
          19 kB
          Hairong Kuang
        4. listFiles1.patch
          19 kB
          Hairong Kuang
        5. listFiles.patch
          17 kB
          Hairong Kuang

          Issue Links

            Activity

              People

              • Assignee:
                hairong Hairong Kuang
                Reporter:
                hairong Hairong Kuang
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: