Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16114

NetUtils#canonicalizeHost gives different value for same host

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.6, 3.1.2
    • Fix Version/s: 3.1.3
    • Component/s: net
    • Labels:
      None
    • Release Note:
      The above patch will resolve the race condition

      Description

      In NetUtils#canonicalizeHost uses ConcurrentHashMap#putIfAbsent to add an entry to the cache

        private static String canonicalizeHost(String host) {
          // check if the host has already been canonicalized
          String fqHost = canonicalizedHostCache.get(host);
          if (fqHost == null) {
            try {
              fqHost = SecurityUtil.getByName(host).getHostName();
              // slight race condition, but won't hurt
              canonicalizedHostCache.putIfAbsent(host, fqHost);
            } catch (UnknownHostException e) {
              fqHost = host;
            }
          }
          return fqHost;
      }
      

       

      If two different threads were invoking this method for the first time (so the cache is empty) and if SecurityUtil#getByName()#getHostName gives two different value for the same host , only one fqHost would be added in the cache and an invalid fqHost would be given to one of the thread which might cause some APIs to fail for the first time `FileSystem#checkPath` even if the path is in the given file system. It might be better if we modify the above method to this

       

        private static String canonicalizeHost(String host) {
          // check if the host has already been canonicalized
          String fqHost = canonicalizedHostCache.get(host);
          if (fqHost == null) {
            try {
              fqHost = SecurityUtil.getByName(host).getHostName();
              // slight race condition, but won't hurt
              canonicalizedHostCache.putIfAbsent(host, fqHost);
              fqHost = canonicalizedHostCache.get(host);
            } catch (UnknownHostException e) {
              fqHost = host;
            }
          }
          return fqHost;
      }
      

       

      So even if other thread get a different host name it will be updated to the cached value.

        Attachments

        1. HADOOP-16114-001.patch
          0.9 kB
          Praveen Krishna

          Activity

            People

            • Assignee:
              praveen_krishna Praveen Krishna
              Reporter:
              praveen_krishna Praveen Krishna

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment