Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6358

Cache the resolved hosts prevent calls to InetAddress.getByName and normalizeHost

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: resourcemanager, security
    • Labels:

      Description

      When running performance tests, we noticed that a lot of time is taken in resolving the host address.
      In our specific scenario, we saw the function org.apache.hadoop.security.SecurityUtil.getInetAddressByName taking a lot of time to resolve the hosts, and the same function is called a lot of times.
      I saw that org.apache.hadoop.yarn.server.resourcemanager.NodesListManager has a cached resolver already because of the same reason.
      So, the proposal is, to make this cache generic and use it to save some time in this functions that we already know, but have it available so the cache can be used anywhere else.

        Attachments

          Activity

            People

            • Assignee:
              jmArreola Jose Miguel Arreola
              Reporter:
              jmArreola Jose Miguel Arreola
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified