[MAPREDUCE-6899] Unable to see logs for Job end notification

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.8.0
• Labels: applicationmaster mapreduce
• Assignee: Unassigned
• Reporter: Satish Saley
• Created: 12/Jun/17 22:06
• Updated: 15/Jun/17 14:22

Description

Hi, I have set mapreduce.job.end-notification.url and I am able to get the notification correctly. But I am unable to see log lines from class org.apache.hadoop.mapreduce.v2.app.MRAppMaster regarding job end notification. When I was trying out different notification urls (before stabilizing my app), I didn't have any way to look at the logs and wasted a lot of time figuring out actual issue. It would be good if these logs are also included in syslog for container.

relevant code

notifyIsLastAMRetry(isLastAMRetry);
      // Stop all services
      // This will also send the final report to the ResourceManager
      LOG.info("Calling stop for all the services"); <---- Able to see this.
      MRAppMaster.this.stop();

      if (isLastAMRetry) {
        // Send job-end notification when it is safe to report termination to
        // users and it is the last AM retry
        if (getConfig().get(MRJobConfig.MR_JOB_END_NOTIFICATION_URL) != null) {
          try {
            LOG.info("Job end notification started for jobID : "
                + job.getReport().getJobId()); <--- unable to see this
            JobEndNotifier notifier = new JobEndNotifier();
            notifier.setConf(getConfig());
            JobReport report = job.getReport();

Is it because we are shutting down all services including JobHistoryEventHandler in MRAppMaster.this.stop(); ?