Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      JobTracker is vulnerable to memory errors/attacks. Few of them are as follows

      • JOB INIT : lot of users submitting large jobs. As every jobs is expanded, the jobtracker's memory can be completely used up
      • JSP : jsp (jobhistory.jsp etc) can also interfere with jobtracker's memory and hence the jobtracker should be protected against such attacks
      • OLD JOBS : lot of completed jobs can garble up jobtracker's memory and hence should be periodically cleaned up. HADOOP-4766 addresses this.

      The main intention of this issue is to track various jira's that help jobtracker battle memory attacks. Jobtracker should always be up and available.

        Issue Links

          Activity

          Hide
          Amar Kamat added a comment -

          As of now these are the cases where the jobtracker should guard itself :

          • JOB INIT : Consider a case when the jobtracker is running low on memory. Ideally we expect the jobtracker to queue(not expand) jobs that might potentially bring down the jobtracker.
          • JSP : loadhistory.jsp caches job analysis results. For larger jobs this can occupy some amount of memory. Job history parsing results are cached because these results are required for fine-grained analysis (like task analysis and job analysis). Parsing the job history again and again for every tip will be costly. The problem occurs when the jobtracker runs low on memory and we try to analyze a job from history.
          • OLD JOBS : Look at HADOOP-4766 for more details.
          Show
          Amar Kamat added a comment - As of now these are the cases where the jobtracker should guard itself : JOB INIT : Consider a case when the jobtracker is running low on memory. Ideally we expect the jobtracker to queue(not expand) jobs that might potentially bring down the jobtracker. JSP : loadhistory.jsp caches job analysis results. For larger jobs this can occupy some amount of memory. Job history parsing results are cached because these results are required for fine-grained analysis (like task analysis and job analysis). Parsing the job history again and again for every tip will be costly. The problem occurs when the jobtracker runs low on memory and we try to analyze a job from history. OLD JOBS : Look at HADOOP-4766 for more details.
          Hide
          Allen Wittenauer added a comment -

          I'm closing this as fixed. There have been a multitude of parameters added that help control the JT's memory usage and with YARN, it's harder for end users to bring the RM down.

          Show
          Allen Wittenauer added a comment - I'm closing this as fixed. There have been a multitude of parameters added that help control the JT's memory usage and with YARN, it's harder for end users to bring the RM down.

            People

            • Assignee:
              Unassigned
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development