Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10616

Improve performance of path handling

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: hdfs
    • Labels:
      None
    • Target Version/s:

      Description

      Path handling in the namesystem and directory is very inefficient. The path is repeatedly resolved, decomposed into path components, recombined to a full path. parsed again, throughout the system. This is directly inefficient for general performance, and indirectly via unnecessary pressure on young gen GC.

      The namesystem should only operate on paths, parse it once into inodes, and the directory should only operate on inodes.

        Attachments

        1. 2.6-2.7.1-heap.png
          75 kB
          Daryn Sharp

          Activity

            People

            • Assignee:
              daryn Daryn Sharp
              Reporter:
              daryn Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

              • Created:
                Updated: