Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2785

MiniMR cluster thread crashes if no hadoop log dir set

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.20.203.0
    • Fix Version/s: None
    • Component/s: jobtracker
    • Labels:
      None
    • Target Version/s:

      Description

      I'm marking this as minor as it is most obvious in the MiniMRCluster, but the root cause is in the JT.

      If you instantiate an MiniMR Cluster without setting hadoop.job.history.location in the configuration and the system property hadoop.log.dir unset, then the JobHistory throws an NPE. In production, that would be picked up as a failure to start the JT. In the MiniMRCluster, all it does is crash the JT thread -which isn't noticed by the MiniMR cluster. You see the logged error, but the tests will just timeout waiting for things to come up

      2011/08/08 17:46:26:427 CEST [ERROR][Thread-44] org.apache.hadoop.mapred.MiniMRCluster - Job tracker crashed <java.lang.NullPointerException> java.lang.NullPointerException
      at java.io.File.<init>(File.java:222)
      at org.apache.hadoop.mapred.JobHistory.initLogDir(JobHistory.java:531)
      at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:499)
      at org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2316)
      at org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2313)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2313)
      at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2171)
      at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300)
      at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner$1.run(MiniMRCluster.java:114)
      at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner$1.run(MiniMRCluster.java:112)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner.run(MiniMRCluster.java:112)
      at java.lang.Thread.run(Thread.java:662)

        Attachments

          Activity

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: