Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2489

Jobsplits with random hostnames can make the queue unusable

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.205.0, 0.23.0
    • 0.20.205.0, 0.23.0
    • jobtracker
    • None
    • Reviewed

    Description

      We saw an issue where a custom InputSplit was returning invalid hostnames for the splits that were then causing the JobTracker to attempt to excessively resolve host names. This caused a major slowdown for the JobTracker. We should prevent invalid InputSplit hostnames from affecting everyone else.

      I propose we implement some verification for the hostnames to try to ensure that we only do DNS lookups on valid hostnames (and fail otherwise). We could also fail the job after a certain number of failures in the resolve.

      Attachments

        1. MAPREDUCE-2489-mapred-v7.patch
          2 kB
          Jeffrey Naisbitt
        2. MAPREDUCE-2489-mapred-v6.patch
          6 kB
          Jeffrey Naisbitt
        3. MAPREDUCE-2489-mapred-v5.patch
          4 kB
          Jeffrey Naisbitt
        4. MAPREDUCE-2489-mapred-v4.patch
          6 kB
          Jeffrey Naisbitt
        5. MAPREDUCE-2489-mapred-v3.patch
          6 kB
          Jeffrey Naisbitt
        6. MAPREDUCE-2489-mapred-v2.patch
          6 kB
          Jeffrey Naisbitt
        7. MAPREDUCE-2489-mapred.patch
          6 kB
          Jeffrey Naisbitt
        8. MAPREDUCE-2489-0.20s-v6.patch
          7 kB
          Jeffrey Naisbitt
        9. MAPREDUCE-2489-0.20s-v5.patch
          9 kB
          Jeffrey Naisbitt
        10. MAPREDUCE-2489-0.20s-v4.patch
          8 kB
          Jeffrey Naisbitt
        11. MAPREDUCE-2489-0.20s-v3.patch
          6 kB
          Jeffrey Naisbitt
        12. MAPREDUCE-2489-0.20s-v2.patch
          16 kB
          Jeffrey Naisbitt
        13. MAPREDUCE-2489-0.20s.patch
          17 kB
          Jeffrey Naisbitt

        Issue Links

        depends upon

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7314 Add support for throwing UnknownHostException when a host doesn't resolve

        • Major - Major loss of function.
        • Closed
        is blocked by

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-7499 Add method for doing a sanity check on hostnames in NetUtils

        • Major - Major loss of function.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            naisbitt Jeffrey Naisbitt
            naisbitt Jeffrey Naisbitt
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment