Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6870

Add FileSystem#listLocatedStatus to list a directory's content together with each file's block locations

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.22.0
    • fs
    • None
    • Reviewed

    Description

      This jira implements the new FileSystem API as proposed in HDFS-202. The new API aims to eliminate individual "getFileBlockLocations" calls to NN for each file in the input directory of a job. Instead, a file's block locations are returned together with FileStatus when listing a directory, thus improving getSplits performance.

      Attachments

        1. listFiles4.patch
          22 kB
          Hairong Kuang
        2. listFiles3.patch
          21 kB
          Hairong Kuang
        3. listFiles2.patch
          19 kB
          Hairong Kuang
        4. listFiles1.patch
          19 kB
          Hairong Kuang
        5. listFiles.patch
          17 kB
          Hairong Kuang

        Issue Links

          Activity

            People

              hairong Hairong Kuang
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: