Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5414

TestTaskAttempt fails jdk7 with NullPointerException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.5-alpha
    • Fix Version/s: 0.23.10, 2.1.1-beta
    • Component/s: test
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails once in a while when i run all of them together.

       
      Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
      Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec <<< FAILURE!
      Results :
      
      Tests in error:
        testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
        testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
        testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
        testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
      
      Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
      

      But if i run a single test case,taking testContainerCleanedWhileRunning for example,it will fail without doubt.

       
       <testcase time="0.057" classname="org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt" name="testContainerCleanedWhileRunning">
          <error type="java.lang.NullPointerException">java.lang.NullPointerException
              at org.apache.hadoop.security.token.Token.write(Token.java:216)
              at org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
              at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
              at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
              at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
              at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
              at org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:601)
              at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
              at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
              at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
              at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
              at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
              at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
              at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
              at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:601)
              at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
              at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
              at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
              at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
              at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
      </error>
          <system-out>2013-07-24 10:32:27,664 INFO  [main] util.RackResolver (RackResolver.java:coreResolve(100)) - Resolved 127.0.0.1 to /default-rack
      2013-07-24 10:32:27,665 INFO  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(1020)) - attempt_1_0002_m_000001_1 TaskAttempt Transitioned from NEW to UNASSIGNED
      2013-07-24 10:32:27,666 INFO  [main] util.RackResolver (RackResolver.java:coreResolve(100)) - Resolved 127.0.0.1 to /default-rack
      2013-07-24 10:32:27,668 INFO  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:createCommonContainerLaunchContext(636)) - Job jar is not present. Not adding any jar to the list of resources.
      2013-07-24 10:32:27,669 INFO  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:createCommonContainerLaunchContext(653)) - The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/root/.staging/job_1_0001/job.xml
      2013-07-24 10:32:27,669 INFO  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:createCommonContainerLaunchContext(675)) - Size of containertokens_dob is 1
      2013-07-24 10:32:27,670 INFO  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:createCommonContainerLaunchContext(685)) - Putting shuffle token in serviceData
      2013-07-24 10:32:27,671 WARN  [main] impl.TaskAttemptImpl (TaskAttemptImpl.java:createCommonContainerLaunchContext(688)) - Cannot locate shuffle secret in credentials. Using job token as shuffle secret.
      </system-out>
      

        Attachments

        1. MAPREDUCE-5414.patch
          4 kB
          Nemon Lou
        2. MAPREDUCE-5414.patch
          3 kB
          Nemon Lou

          Activity

            People

            • Assignee:
              nemon Nemon Lou
              Reporter:
              nemon Nemon Lou
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: