Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1964

TestKillQuery::testKillQueryFromInitState causes NPE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0, 0.11.1
    • Component/s: QueryMaster
    • Labels:
      None

      Description

      If a task status is new and a query receive kill message, TaskAttempt will be null.

      2015-11-02 21:13:16,126 INFO: org.apache.tajo.querymaster.Query (handle(868)) - Processing q_1446498795879_0000 of type START
      2015-11-02 21:13:16,128 INFO: org.apache.tajo.engine.planner.global.ParallelExecutionQueue (first(84)) - Initial executable blocks [eb_1446498795879_0000_000003]
      2015-11-02 21:13:16,129 INFO: org.apache.tajo.querymaster.Query (handle(886)) - q_1446498795879_0000 Query Transitioned from QUERY_NEW to QUERY_RUNNING
      2015-11-02 21:13:16,129 INFO: org.apache.tajo.querymaster.Stage (calculateShuffleOutputNum(1026)) - ============>>>>> Unexpected Case! <<<<<================
      2015-11-02 21:13:16,129 INFO: org.apache.tajo.querymaster.Stage (calculateShuffleOutputNum(1030)) - eb_1446498795879_0000_000003, Table's volume is approximately 1 MB
      2015-11-02 21:13:16,129 INFO: org.apache.tajo.querymaster.Stage (calculateShuffleOutputNum(1033)) - eb_1446498795879_0000_000003, The determined number of partitions is 1
      2015-11-02 21:13:16,150 INFO: org.apache.tajo.querymaster.Stage (initTaskScheduler(901)) - org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1446498795879_0000_000003
      2015-11-02 21:13:16,151 INFO: org.apache.tajo.storage.FileTablespace (getSplits(564)) - Total # of splits: 1
      2015-11-02 21:13:16,152 INFO: org.apache.tajo.storage.FileTablespace (getSplits(564)) - Total # of splits: 1
      2015-11-02 21:13:16,152 INFO: org.apache.tajo.querymaster.Repartitioner (scheduleFragmentsForJoinQuery(230)) - [Distributed Join Strategy] : Broadcast Join, base_table=default.t2, base_volume=724
      2015-11-02 21:13:16,152 INFO: org.apache.tajo.storage.FileTablespace (getSplits(564)) - Total # of splits: 1
      2015-11-02 21:13:16,153 INFO: org.apache.tajo.querymaster.Query (handle(868)) - Processing q_1446498795879_0000 of type KILL
      2015-11-02 21:13:16,153 INFO: org.apache.tajo.querymaster.Query (handle(886)) - q_1446498795879_0000 Query Transitioned from QUERY_RUNNING to QUERY_KILL_WAIT
      2015-11-02 21:13:16,153 INFO: org.apache.tajo.storage.FileTablespace (getSplits(564)) - Total # of splits: 1
      2015-11-02 21:13:16,153 INFO: org.apache.tajo.querymaster.DefaultTaskScheduler (stop(163)) - Task Scheduler stopped
      2015-11-02 21:13:16,153 INFO: org.apache.tajo.querymaster.Stage (run(853)) - 1 objects are scheduled
      2015-11-02 21:13:16,155 FATAL: org.apache.hadoop.yarn.event.AsyncDispatcher (dispatch(181)) - Error in dispatcher thread
      java.lang.NullPointerException
      	at org.apache.tajo.querymaster.DefaultTaskScheduler.releaseTaskAttempt(DefaultTaskScheduler.java:332)
      	at org.apache.tajo.querymaster.Stage$TaskCompletedTransition.transition(Stage.java:1228)
      	at org.apache.tajo.querymaster.Stage$TaskCompletedTransition.transition(Stage.java:1212)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
      	at org.apache.tajo.querymaster.Stage.handle(Stage.java:801)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:213)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:207)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
      	at org.apache.tajo.querymaster.TestKillQuery$MockAsyncDispatch.dispatch(TestKillQuery.java:305)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
      	at java.lang.Thread.run(Thread.java:745)
      2015-11-02 21:13:16,156 INFO: org.apache.tajo.querymaster.Stage (transition(1407)) - Stage completed - eb_1446498795879_0000_000003 (total=1, success=0, killed=0)
      2015-11-02 21:13:16,157 ERROR: org.apache.tajo.querymaster.Stage (transition(1430)) - 
      java.lang.NullPointerException
      	at org.apache.tajo.querymaster.Stage.computeStatFromTasks(Stage.java:697)
      	at org.apache.tajo.querymaster.Stage.finalizeStats(Stage.java:771)
      	at org.apache.tajo.querymaster.Stage.complete(Stage.java:532)
      	at org.apache.tajo.querymaster.Stage$StageCompleteTransition.transition(Stage.java:1426)
      	at org.apache.tajo.querymaster.Stage$StageCompleteTransition.transition(Stage.java:1399)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
      	at org.apache.tajo.querymaster.Stage.handle(Stage.java:801)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:213)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:207)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
      	at org.apache.tajo.querymaster.TestKillQuery$MockAsyncDispatch.dispatch(TestKillQuery.java:305)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
      	at java.lang.Thread.run(Thread.java:745)
      2015-11-02 21:13:16,157 INFO: org.apache.tajo.querymaster.DefaultTaskScheduler (stop(163)) - Task Scheduler stopped
      2015-11-02 21:13:16,158 ERROR: org.apache.tajo.querymaster.Stage (handle(804)) - Can't handle this event at current state, eventType:SQ_STAGE_COMPLETED, oldState:KILL_WAIT, nextState:KILL_WAIT
      org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: SQ_STAGE_COMPLETED at KILL_WAIT
      	at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:388)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
      	at org.apache.tajo.querymaster.Stage.handle(Stage.java:801)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:213)
      	at org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher.handle(QueryMasterTask.java:207)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
      	at org.apache.tajo.querymaster.TestKillQuery$MockAsyncDispatch.dispatch(TestKillQuery.java:305)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
      	at java.lang.Thread.run(Thread.java:745)
      2015-11-02 21:13:16,158 INFO: org.apache.tajo.querymaster.Query (handle(868)) - Processing q_1446498795879_0000 of type STAGE_COMPLETED
      2015-11-02 21:13:16,158 INFO: org.apache.tajo.querymaster.Query (transition(815)) - Complete Stage[eb_1446498795879_0000_000003], State: ERROR, 1/1. 
      2015-11-02 21:13:16,158 INFO: org.apache.tajo.querymaster.DefaultTaskScheduler (stop(163)) - Task Scheduler stopped
      2015-11-02 21:13:16,158 INFO: org.apache.tajo.querymaster.Query (handle(868)) - Processing q_1446498795879_0000 of type QUERY_COMPLETED
      2015-11-02 21:13:16,158 INFO: org.apache.tajo.querymaster.Query (handle(886)) - q_1446498795879_0000 Query Transitioned from QUERY_KILL_WAIT to QUERY_ERROR
      2015-11-02 21:13:16,159 INFO: org.apache.tajo.querymaster.Query (handle(868)) - Processing q_1446498795879_0000 of type STAGE_COMPLETED
      2015-11-02 21:13:16,159 INFO: org.apache.tajo.querymaster.QueryMasterTask (handle(296)) - Query completion notified from q_1446498795879_0000 final state: QUERY_ERROR
      2015-11-02 21:13:16,159 WARN: org.apache.tajo.querymaster.QueryMaster (stopQuery(283)) - No query info:q_1446498795879_0000
      java.io.IOException: QUERY_ERROR
      	at org.apache.tajo.TajoTestingCluster.waitForQueryState(TajoTestingCluster.java:751)
      	at org.apache.tajo.querymaster.TestKillQuery.testKillQueryFromInitState(TestKillQuery.java:135)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
      Stage: [eb_1446498795879_0000_000003] (Total: 1, Complete: 1, Success: 0, Killed: 0, Failed: 0)
      2015-11-02 21:13:26,208 INFO: org.apache.tajo.querymaster.QueryMasterTask (serviceStop(172)) - Stopping QueryMasterTask:q_1446498795879_0000
      2015-11-02 21:13:26,209 INFO: org.apache.tajo.querymaster.QueryMasterTask (cleanupQuery(473)) - Cleanup resources of all workers. Query: q_1446498795879_0000, workers: 0
      2015-11-02 21:13:26,212 INFO: org.apache.tajo.querymaster.QueryMasterTask (serviceStop(188)) - Stopped QueryMasterTask:q_1446498795879_0000
      2015-11-02 21:13:26,222 INFO: org.apache.tajo.session.SessionManager (removeSession(85)) - Session b628eaf7-768d-43aa-bd48-82247ae9b44c is removed.
      

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/847

        TAJO-1964: TestKillQuery::testKillQueryFromInitState causes NPE.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-1964

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/847.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #847


        commit f8263904f242e116999bc2282b5280f5685f7b00
        Author: Jinho Kim <jhkim@apache.org>
        Date: 2015-11-05T09:15:14Z

        TAJO-1964: TestKillQuery::testKillQueryFromInitState causes NPE.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/847 TAJO-1964 : TestKillQuery::testKillQueryFromInitState causes NPE. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1964 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/847.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #847 commit f8263904f242e116999bc2282b5280f5685f7b00 Author: Jinho Kim <jhkim@apache.org> Date: 2015-11-05T09:15:14Z TAJO-1964 : TestKillQuery::testKillQueryFromInitState causes NPE.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/847#issuecomment-154270891

        +1. I've manually tested with 'mvn install'.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/847#issuecomment-154270891 +1. I've manually tested with 'mvn install'.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/847

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/847
        Hide
        jhkim Jinho Kim added a comment -

        committed it
        Thanks

        Show
        jhkim Jinho Kim added a comment - committed it Thanks
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #584 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/584/)
        TAJO-1964: TestKillQuery::testKillQueryFromInitState causes NPE. (jhkim: rev 6c3ae723f4613d36d05a76d01cd394aa788857ba)

        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #584 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/584/ ) TAJO-1964 : TestKillQuery::testKillQueryFromInitState causes NPE. (jhkim: rev 6c3ae723f4613d36d05a76d01cd394aa788857ba) CHANGES tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-build #964 (See https://builds.apache.org/job/Tajo-master-build/964/)
        TAJO-1964: TestKillQuery::testKillQueryFromInitState causes NPE. (jhkim: rev 6c3ae723f4613d36d05a76d01cd394aa788857ba)

        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        • CHANGES
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #964 (See https://builds.apache.org/job/Tajo-master-build/964/ ) TAJO-1964 : TestKillQuery::testKillQueryFromInitState causes NPE. (jhkim: rev 6c3ae723f4613d36d05a76d01cd394aa788857ba) tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java CHANGES

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development