Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3770

[Rumen] Zombie.getJobConf() results into NPE

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 0.23.1
    • Component/s: tools/rumen
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Tags:
      rumen job-name
    • Target Version/s:

      Description

      The error trace is as follows

      java.lang.NullPointerException
              at java.util.Hashtable.put(Hashtable.java:394)
              at java.util.Properties.setProperty(Properties.java:143)
              at org.apache.hadoop.conf.Configuration.set(Configuration.java:623)
              at org.apache.hadoop.mapred.JobConf.setJobName(JobConf.java:1322)
              at org.apache.hadoop.tools.rumen.ZombieJob.getJobConf(ZombieJob.java:139)
              at org.apache.hadoop.mapred.gridmix.DistributedCacheEmulator.updateHDFSDistCacheFilesList(DistributedCacheEmulator.java:315)
              at org.apache.hadoop.mapred.gridmix.DistributedCacheEmulator.buildDistCacheFilesList(DistributedCacheEmulator.java:280)
              at org.apache.hadoop.mapred.gridmix.DistributedCacheEmulator.setupGenerateDistCacheData(DistributedCacheEmulator.java:253)
              at org.apache.hadoop.mapred.gridmix.Gridmix.setupDistCacheEmulation(Gridmix.java:528)
              at org.apache.hadoop.mapred.gridmix.Gridmix.setupEmulation(Gridmix.java:501)
              at org.apache.hadoop.mapred.gridmix.Gridmix.start(Gridmix.java:433)
              at org.apache.hadoop.mapred.gridmix.Gridmix.runJob(Gridmix.java:380)
              at org.apache.hadoop.mapred.gridmix.Gridmix.access$000(Gridmix.java:56)
              at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:313)
              at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:311)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:396)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
              at org.apache.hadoop.mapred.gridmix.Gridmix.run(Gridmix.java:311)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
              at org.apache.hadoop.mapred.gridmix.Gridmix.main(Gridmix.java:606)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
      

      The bug seems to be in ZombieJob#getName() where a not-null check for jobName.getValue() is missing.

        Issue Links

          Activity

          Hide
          Vinod Kumar Vavilapalli added a comment -

          This was meant to be merged into 0.23.1 but wasn't. It was only on trunk and branch-0.23. I just merged it into 0.23.1.

          Show
          Vinod Kumar Vavilapalli added a comment - This was meant to be merged into 0.23.1 but wasn't. It was only on trunk and branch-0.23. I just merged it into 0.23.1.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #983 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/983/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #983 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/983/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #185 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/185/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #185 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/185/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #163 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/163/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #163 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/163/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #950 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/950/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #950 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/950/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #528 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/528/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #528 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/528/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #524 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/524/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #524 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/524/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/513/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/513/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241865 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Amar Kamat added a comment -

          I just committed this to trunk and branch-0.23.

          Show
          Amar Kamat added a comment - I just committed this to trunk and branch-0.23.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1707 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1707/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1707 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1707/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1696 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1696/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1696 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1696/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1771 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1771/)
          MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)

          amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1771 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1771/ ) MAPREDUCE-3770 . Zombie.getJobConf() results into NPE. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241845 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ZombieJob.java
          Hide
          Ravi Gummadi added a comment -

          Code change looks fine to me.
          +1, though it would be nice to have some testcase.

          Show
          Ravi Gummadi added a comment - Code change looks fine to me. +1, though it would be nice to have some testcase.
          Hide
          Amar Kamat added a comment -

          Attaching a patch which takes care of the NPE. This NPE can occur when the trace contains jobs with no job-name i.e old job history files.

          Show
          Amar Kamat added a comment - Attaching a patch which takes care of the NPE. This NPE can occur when the trace contains jobs with no job-name i.e old job history files.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This is needed for 0.23.1.

          Show
          Vinod Kumar Vavilapalli added a comment - This is needed for 0.23.1.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Amar, this should be simple enough fix, can you please take this to completion including any tests you may want to add? Thanks.

          Show
          Vinod Kumar Vavilapalli added a comment - Amar, this should be simple enough fix, can you please take this to completion including any tests you may want to add? Thanks.

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development