Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2489

Jobsplits with random hostnames can make the queue unusable

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.205.0, 0.23.0
    • 0.20.205.0, 0.23.0
    • jobtracker
    • None
    • Reviewed

    Description

      We saw an issue where a custom InputSplit was returning invalid hostnames for the splits that were then causing the JobTracker to attempt to excessively resolve host names. This caused a major slowdown for the JobTracker. We should prevent invalid InputSplit hostnames from affecting everyone else.

      I propose we implement some verification for the hostnames to try to ensure that we only do DNS lookups on valid hostnames (and fail otherwise). We could also fail the job after a certain number of failures in the resolve.

      Attachments

        1. MAPREDUCE-2489-0.20s.patch
          17 kB
          Jeffrey Naisbitt
        2. MAPREDUCE-2489-0.20s-v2.patch
          16 kB
          Jeffrey Naisbitt
        3. MAPREDUCE-2489-0.20s-v3.patch
          6 kB
          Jeffrey Naisbitt
        4. MAPREDUCE-2489-0.20s-v4.patch
          8 kB
          Jeffrey Naisbitt
        5. MAPREDUCE-2489-0.20s-v5.patch
          9 kB
          Jeffrey Naisbitt
        6. MAPREDUCE-2489-0.20s-v6.patch
          7 kB
          Jeffrey Naisbitt
        7. MAPREDUCE-2489-mapred.patch
          6 kB
          Jeffrey Naisbitt
        8. MAPREDUCE-2489-mapred-v2.patch
          6 kB
          Jeffrey Naisbitt
        9. MAPREDUCE-2489-mapred-v3.patch
          6 kB
          Jeffrey Naisbitt
        10. MAPREDUCE-2489-mapred-v4.patch
          6 kB
          Jeffrey Naisbitt
        11. MAPREDUCE-2489-mapred-v5.patch
          4 kB
          Jeffrey Naisbitt
        12. MAPREDUCE-2489-mapred-v6.patch
          6 kB
          Jeffrey Naisbitt
        13. MAPREDUCE-2489-mapred-v7.patch
          2 kB
          Jeffrey Naisbitt

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7314 Add support for throwing UnknownHostException when a host doesn't resolve

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-7499 Add method for doing a sanity check on hostnames in NetUtils

          • Major - Major loss of function.
          • Closed

          Activity

            People

              naisbitt Jeffrey Naisbitt
              naisbitt Jeffrey Naisbitt
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: