Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1981

Improve getSplits performance by using listLocatedStatus

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 0.23.10, 2.1.1-beta
    • job submission
    • None
    • Reviewed

    Description

      This jira will make FileInputFormat and CombinedFileInputForm to use the new API, thus reducing the number of RPCs to HDFS NameNode.

      Attachments

        1. mapredListFiles.patch
          16 kB
          Hairong Kuang
        2. mapredListFiles1.patch
          15 kB
          Hairong Kuang
        3. mapredListFiles2.patch
          15 kB
          Hairong Kuang
        4. mapredListFiles3.patch
          14 kB
          Hairong Kuang
        5. mapredListFiles4.patch
          15 kB
          Hairong Kuang
        6. mapredListFiles5.patch
          14 kB
          Hairong Kuang
        7. MAPREDUCE-1981.branch-0.23.patch
          26 kB
          Jason Darrell Lowe
        8. MAPREDUCE-1981.patch
          26 kB
          Jason Darrell Lowe

        Issue Links

          Activity

            People

              hairong Hairong Kuang
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: