Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6467

Performance improvement for liststatus on directories in hadoop archives.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      A liststatus call on a directory in hadoop archives leads to ( 2* number of files in directory) open calls to the namenode. This is very sub optimal and needs to be fixed to make it performant enough to be used on a daily basis.

        Attachments

        1. Archives_performance.docx
          111 kB
          Mahadev konar
        2. Archives_performance.docx
          94 kB
          Mahadev konar
        3. HADOOP-6467_v3.patch
          4 kB
          Mahadev konar
        4. HADOOP-6467.patch
          4 kB
          Mahadev konar
        5. HADOOP-6467.patch
          7 kB
          Mahadev konar
        6. HADOOP-6467.patch
          6 kB
          Mahadev konar
        7. HADOOP-6467-v2.patch
          4 kB
          Mahadev konar
        8. HADOOP-6467-y.0.20-branch-v2.patch
          5 kB
          Mahadev konar
        9. HADOOP-6467-y.0.20-branch-v2.patch
          4 kB
          Mahadev konar
        10. HADOOP-6467-y0.20-branch.patch
          4 kB
          Mahadev konar

          Issue Links

            Activity

              People

              • Assignee:
                mahadev Mahadev konar
                Reporter:
                mahadev Mahadev konar
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: