Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4464

Reduce tasks failing with NullPointerException in ConcurrentHashMap.get()

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.2.0
    • Component/s: task
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If DNS does not resolve hostnames properly, reduce tasks can fail with a very misleading exception.

      as per my peer Ahmed's diagnosis:

      In ReduceTask, it seems that event.getTaskTrackerHttp() returns a malformed URI, and so host from:

      String host = u.getHost();
      

      is evaluated to null and the NullPointerException is thrown afterwards in the ConcurrentHashMap.

      I have written a patch to check for a null hostname condition when getHost is called in the getMapCompletionEvents method and print an intelligible warning message rather than suppressing it until later when it becomes confusing and misleading.

        Attachments

        1. MAPREDUCE-4464.patch
          0.9 kB
          Harsh J
        2. MAPREDUCE-4464_new.patch
          0.8 kB
          Clint Heath
        3. MAPREDUCE-4464.patch
          0.8 kB
          Clint Heath

          Activity

            People

            • Assignee:
              heathcd Clint Heath
              Reporter:
              heathcd Clint Heath
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified