Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6899

Unable to see logs for Job end notification

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.8.0
    • Fix Version/s: None
    • Component/s: None

      Description

      Hi, I have set mapreduce.job.end-notification.url and I am able to get the notification correctly. But I am unable to see log lines from class org.apache.hadoop.mapreduce.v2.app.MRAppMaster regarding job end notification. When I was trying out different notification urls (before stabilizing my app), I didn't have any way to look at the logs and wasted a lot of time figuring out actual issue. It would be good if these logs are also included in syslog for container.

      relevant code

      notifyIsLastAMRetry(isLastAMRetry);
            // Stop all services
            // This will also send the final report to the ResourceManager
            LOG.info("Calling stop for all the services"); <---- Able to see this.
            MRAppMaster.this.stop();
      
            if (isLastAMRetry) {
              // Send job-end notification when it is safe to report termination to
              // users and it is the last AM retry
              if (getConfig().get(MRJobConfig.MR_JOB_END_NOTIFICATION_URL) != null) {
                try {
                  LOG.info("Job end notification started for jobID : "
                      + job.getReport().getJobId()); <--- unable to see this
                  JobEndNotifier notifier = new JobEndNotifier();
                  notifier.setConf(getConfig());
                  JobReport report = job.getReport();
      

      Is it because we are shutting down all services including JobHistoryEventHandler in MRAppMaster.this.stop(); ?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              satishsaley Satish Subhashrao Saley
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: