Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4464

Reduce tasks failing with NullPointerException in ConcurrentHashMap.get()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 1.0.0
    • 1.2.0
    • task
    • None
    • Reviewed

    Description

      If DNS does not resolve hostnames properly, reduce tasks can fail with a very misleading exception.

      as per my peer Ahmed's diagnosis:

      In ReduceTask, it seems that event.getTaskTrackerHttp() returns a malformed URI, and so host from:

      String host = u.getHost();
      

      is evaluated to null and the NullPointerException is thrown afterwards in the ConcurrentHashMap.

      I have written a patch to check for a null hostname condition when getHost is called in the getMapCompletionEvents method and print an intelligible warning message rather than suppressing it until later when it becomes confusing and misleading.

      Attachments

        1. MAPREDUCE-4464.patch
          0.8 kB
          Clint Heath
        2. MAPREDUCE-4464.patch
          0.9 kB
          Harsh J
        3. MAPREDUCE-4464_new.patch
          0.8 kB
          Clint Heath

        Activity

          People

            heathcd Clint Heath
            heathcd Clint Heath
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified