Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13390

GC pressure of NetworkTopology.isAncestor call can be eliminated

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.6.0, 2.7.0, 2.8.0
    • Fix Version/s: None
    • Component/s: net
    • Labels:
      None

      Description

      NetworkTopology.isAncestor is called in NetworkTopology.getLeaf, which is excessively used in block placement policies (chooseRandom for example).

      Currently, the implementation calls getPath twice. And getPath performs string concatenation on the fly. On a busy NameNode, this introduces more GC pressure and CPU overhead for block allocation.
      Given that network location and node name does not generally change frequently, we can cache path as a properly of Node and update accordingly while network location or node name is changing.
      Also, one of these getPath calls in getLeaf can be eliminated because we are expecting identical result in both calls.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              He Tianyi He Tianyi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: