Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-819

KillQuery does not work query occasionally.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0, 0.8.1
    • Component/s: QueryMaster, Worker
    • Labels:
      None

      Description

      There are two types of killing queries. First is that QueryMaster cancel QueryUnitAttempt before QueryUnitAttempt be passed to TajoWorker. Second is that TajoWorker cancel running QueryUnitAttempt.

      If QueryMaster cancel QueryUnitAttempts, it will be stopped. But unfortunately, at nearly the same time, a few QueryUnitAttempts can be passed to TajoWorker. And these never will be stopped because it can't find valid state for state machines.

      1. TAJO-819.patch
        18 kB
        Jaehwa Jung
      2. TAJO-819_3.patch
        26 kB
        Jaehwa Jung
      3. TAJO-819_2.patch
        18 kB
        Jaehwa Jung
      4. killQuery.png
        40 kB
        Jaehwa Jung

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-build #221 (See https://builds.apache.org/job/Tajo-master-build/221/)
          TAJO-819: KillQuery does not work for running query on TajoWorker. (jaehwa) (jhjung: rev 98f142cc73db1449ec04353c90897721806ecd42)

          • tajo-core/src/test/java/org/apache/tajo/master/rm/TestTajoResourceManager.java
          • tajo-core/src/main/java/org/apache/tajo/webapp/QueryExecutorServlet.java
          • tajo-core/src/main/resources/webapps/admin/query.jsp
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
          • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/TajoRMContext.java
          • CHANGES
          • tajo-core/src/main/java/org/apache/tajo/master/rm/Worker.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoAdmin.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #221 (See https://builds.apache.org/job/Tajo-master-build/221/ ) TAJO-819 : KillQuery does not work for running query on TajoWorker. (jaehwa) (jhjung: rev 98f142cc73db1449ec04353c90897721806ecd42) tajo-core/src/test/java/org/apache/tajo/master/rm/TestTajoResourceManager.java tajo-core/src/main/java/org/apache/tajo/webapp/QueryExecutorServlet.java tajo-core/src/main/resources/webapps/admin/query.jsp tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java tajo-core/src/main/java/org/apache/tajo/master/rm/TajoRMContext.java CHANGES tajo-core/src/main/java/org/apache/tajo/master/rm/Worker.java tajo-client/src/main/java/org/apache/tajo/client/TajoAdmin.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-0.8.1-build #101 (See https://builds.apache.org/job/Tajo-0.8.1-build/101/)
          TAJO-819: KillQuery does not work for running query on TajoWorker. (jaehwa) (jhjung: rev 590cf8464f6c1f4c12a0856a5c2ab698c548fac8)

          • tajo-client/src/main/java/org/apache/tajo/client/TajoAdmin.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
          • CHANGES
          • tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java
          • tajo-core/src/main/resources/webapps/admin/query.jsp
          • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
          • tajo-core/src/main/java/org/apache/tajo/webapp/QueryExecutorServlet.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.8.1-build #101 (See https://builds.apache.org/job/Tajo-0.8.1-build/101/ ) TAJO-819 : KillQuery does not work for running query on TajoWorker. (jaehwa) (jhjung: rev 590cf8464f6c1f4c12a0856a5c2ab698c548fac8) tajo-client/src/main/java/org/apache/tajo/client/TajoAdmin.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java CHANGES tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java tajo-core/src/main/resources/webapps/admin/query.jsp tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-core/src/main/java/org/apache/tajo/webapp/QueryExecutorServlet.java
          Hide
          blrunner Jaehwa Jung added a comment -

          This patch got +1 at RB.
          I've just committed it to master and branch-0.8.1.

          Show
          blrunner Jaehwa Jung added a comment - This patch got +1 at RB. I've just committed it to master and branch-0.8.1.
          Hide
          blrunner Jaehwa Jung added a comment - - edited

          I added a button to kill running queries at query.jsp.
          You can find it at 'http://$

          {tajo.master.info-http.address}

          :26080/query.jsp'

          P.S I attached a screen capture file.

          Show
          blrunner Jaehwa Jung added a comment - - edited I added a button to kill running queries at query.jsp. You can find it at 'http://$ {tajo.master.info-http.address} :26080/query.jsp' P.S I attached a screen capture file.
          Hide
          blrunner Jaehwa Jung added a comment -

          I updated the patch as follows:

          • Update TajoClient::killQuery returning type.
          • Add kill button at query.jsp
          • Add kill condition at FetchRunner
          Show
          blrunner Jaehwa Jung added a comment - I updated the patch as follows: Update TajoClient::killQuery returning type. Add kill button at query.jsp Add kill condition at FetchRunner
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645736/TAJO-819_2.patch
          against master revision d0528d1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 195 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/427//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/427//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/427//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645736/TAJO-819_2.patch against master revision d0528d1. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 195 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/427//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/427//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/427//console This message is automatically generated.
          Hide
          blrunner Jaehwa Jung added a comment -

          Updated the review request against branch master in reviewboard
          https://reviews.apache.org/r/21325/

          Show
          blrunner Jaehwa Jung added a comment - Updated the review request against branch master in reviewboard https://reviews.apache.org/r/21325/
          Hide
          blrunner Jaehwa Jung added a comment - - edited

          This patch includes a few features as follows:

          • Added transition for QueryUnit and QueryUnitAttempt
          • Added transition type for debugging
          • Check stopped QueryId for blocking unexpected resource requests
          Show
          blrunner Jaehwa Jung added a comment - - edited This patch includes a few features as follows: Added transition for QueryUnit and QueryUnitAttempt Added transition type for debugging Check stopped QueryId for blocking unexpected resource requests
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12644445/TAJO-819.patch
          against master revision 77fdbfb.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 187 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/421//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/421//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/421//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12644445/TAJO-819.patch against master revision 77fdbfb. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 187 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/421//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/421//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/421//console This message is automatically generated.
          Hide
          blrunner Jaehwa Jung added a comment -

          This bug is very deeply related to TAJO-818.

          Show
          blrunner Jaehwa Jung added a comment - This bug is very deeply related to TAJO-818 .
          Hide
          blrunner Jaehwa Jung added a comment -

          Created a review request against branch master in reviewboard.
          https://reviews.apache.org/r/21325/

          Show
          blrunner Jaehwa Jung added a comment - Created a review request against branch master in reviewboard. https://reviews.apache.org/r/21325/

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              blrunner Jaehwa Jung
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development