Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-2048

QueryMaster and TajoWorker should support the exception propagation

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0, 0.11.1
    • Component/s: QueryMaster, Worker
    • Labels:
      None

      Description

      Error propagation system was refactored by TAJO-1625. But worker and query master is missing in TAJO-1625.
      This issue improve error propagation system in TajoWorker and QueryMaster

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/936

        TAJO-2048: QueryMaster and TajoWorker should support the exception propagation.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-2048

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/936.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #936


        commit 1c725cf071247d04b59bfe73f2e28adcfa489be3
        Author: Jinho Kim <jhkim@apache.org>
        Date: 2016-01-11T04:46:14Z

        TAJO-2048: QueryMaster and TajoWorker should support the exception propagation.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/936 TAJO-2048 : QueryMaster and TajoWorker should support the exception propagation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-2048 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/936.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #936 commit 1c725cf071247d04b59bfe73f2e28adcfa489be3 Author: Jinho Kim <jhkim@apache.org> Date: 2016-01-11T04:46:14Z TAJO-2048 : QueryMaster and TajoWorker should support the exception propagation.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/936#discussion_r50334735

        — Diff: tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java —
        @@ -1415,8 +1434,9 @@ public StageState transition(Stage stage, StageEvent stageEvent)

        { stage.abort(StageState.KILLED); return StageState.KILLED; }

        else {

        • LOG.error("Invalid State " + stage.getSynchronizedState() + " State");
        • stage.abort(StageState.ERROR);
          + TajoInternalError error = new TajoInternalError("Invalid State " + stage.getSynchronizedState() + " State");
            • End diff –

        I wonder when this else clause can be executed. Would you explain some details?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/936#discussion_r50334735 — Diff: tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java — @@ -1415,8 +1434,9 @@ public StageState transition(Stage stage, StageEvent stageEvent) { stage.abort(StageState.KILLED); return StageState.KILLED; } else { LOG.error("Invalid State " + stage.getSynchronizedState() + " State"); stage.abort(StageState.ERROR); + TajoInternalError error = new TajoInternalError("Invalid State " + stage.getSynchronizedState() + " State"); End diff – I wonder when this else clause can be executed. Would you explain some details?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/936#discussion_r50335860

        — Diff: tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java —
        @@ -505,6 +503,11 @@ public GetQueryStatusResponse getQueryStatus(RpcController controller, GetQueryS
        builder.setFinishTime(queryInfo.getFinishTime());
        } else {
        builder.setFinishTime(System.currentTimeMillis());
        +
        + if(!StringUtils.isEmpty(queryInfo.getLastMessage()))

        { + builder.setErrorMessage(queryInfo.getLastMessage()); + }

        + builder.setState(OK);
        — End diff –

        It seems duplicate code to set the query state.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/936#discussion_r50335860 — Diff: tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java — @@ -505,6 +503,11 @@ public GetQueryStatusResponse getQueryStatus(RpcController controller, GetQueryS builder.setFinishTime(queryInfo.getFinishTime()); } else { builder.setFinishTime(System.currentTimeMillis()); + + if(!StringUtils.isEmpty(queryInfo.getLastMessage())) { + builder.setErrorMessage(queryInfo.getLastMessage()); + } + builder.setState(OK); — End diff – It seems duplicate code to set the query state.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/936#issuecomment-173394473

        Hi @jinossy, thanks for your work. It looks good overall.
        I left a couple of comments. Please consider them.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/936#issuecomment-173394473 Hi @jinossy, thanks for your work. It looks good overall. I left a couple of comments. Please consider them.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/936#discussion_r50494584

        — Diff: tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java —
        @@ -1415,8 +1434,9 @@ public StageState transition(Stage stage, StageEvent stageEvent)

        { stage.abort(StageState.KILLED); return StageState.KILLED; }

        else {

        • LOG.error("Invalid State " + stage.getSynchronizedState() + " State");
        • stage.abort(StageState.ERROR);
          + TajoInternalError error = new TajoInternalError("Invalid State " + stage.getSynchronizedState() + " State");
            • End diff –

        @jihoonson
        this else clause seems to never be executed. I will remove this code block for clarify

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on a diff in the pull request: https://github.com/apache/tajo/pull/936#discussion_r50494584 — Diff: tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java — @@ -1415,8 +1434,9 @@ public StageState transition(Stage stage, StageEvent stageEvent) { stage.abort(StageState.KILLED); return StageState.KILLED; } else { LOG.error("Invalid State " + stage.getSynchronizedState() + " State"); stage.abort(StageState.ERROR); + TajoInternalError error = new TajoInternalError("Invalid State " + stage.getSynchronizedState() + " State"); End diff – @jihoonson this else clause seems to never be executed. I will remove this code block for clarify
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/936#issuecomment-173812292

        +1 ship it!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/936#issuecomment-173812292 +1 ship it!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #660 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/660/)
        TAJO-2048: QueryMaster and TajoWorker should support the exception (jhkim: rev b8b7066a23d53dbf4456e15a0a7dd18162cb8254)

        • tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        • CHANGES
        • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java
        • tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java
        • tajo-core/src/main/proto/ResourceProtos.proto
        • tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
        • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #660 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/660/ ) TAJO-2048 : QueryMaster and TajoWorker should support the exception (jhkim: rev b8b7066a23d53dbf4456e15a0a7dd18162cb8254) tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java CHANGES tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java tajo-core/src/main/proto/ResourceProtos.proto tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/936

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/936
        Hide
        jhkim Jinho Kim added a comment -

        committed it.
        Thanks for your review!

        Show
        jhkim Jinho Kim added a comment - committed it. Thanks for your review!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #1060 (See https://builds.apache.org/job/Tajo-master-build/1060/)
        TAJO-2048: QueryMaster and TajoWorker should support the exception (jhkim: rev b8b7066a23d53dbf4456e15a0a7dd18162cb8254)

        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java
        • tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java
        • tajo-core/src/main/proto/ResourceProtos.proto
        • tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java
        • tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java
        • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #1060 (See https://builds.apache.org/job/Tajo-master-build/1060/ ) TAJO-2048 : QueryMaster and TajoWorker should support the exception (jhkim: rev b8b7066a23d53dbf4456e15a0a7dd18162cb8254) tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java CHANGES tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java tajo-core/src/main/proto/ResourceProtos.proto tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-0.11.1-build #156 (See https://builds.apache.org/job/Tajo-0.11.1-build/156/)
        TAJO-2048: QueryMaster and TajoWorker should support the exception (jhkim: rev 031cf3205c135e43a8e944db19a50797138c26c4)

        • tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
        • tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java
        • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java
        • tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java
        • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java
        • tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java
        • tajo-core/src/main/proto/ResourceProtos.proto
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.11.1-build #156 (See https://builds.apache.org/job/Tajo-0.11.1-build/156/ ) TAJO-2048 : QueryMaster and TajoWorker should support the exception (jhkim: rev 031cf3205c135e43a8e944db19a50797138c26c4) tajo-common/src/main/java/org/apache/tajo/exception/ErrorUtil.java tajo-core/src/main/java/org/apache/tajo/querymaster/Task.java tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskFatalErrorEvent.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCliNegatives.java tajo-core/src/main/java/org/apache/tajo/master/event/TaskTAttemptFailedEvent.java tajo-core/src/main/java/org/apache/tajo/worker/TaskContainer.java tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java tajo-core/src/main/java/org/apache/tajo/master/QueryManager.java tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java tajo-core/src/main/java/org/apache/tajo/master/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/RangeShuffleFileWriteExec.java tajo-core/src/main/java/org/apache/tajo/master/event/StageTaskFailedEvent.java tajo-core/src/main/java/org/apache/tajo/querymaster/TaskAttempt.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-core/src/main/java/org/apache/tajo/worker/TaskImpl.java tajo-core-tests/src/test/java/org/apache/tajo/worker/MockExecutionBlock.java CHANGES tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/worker/TaskExecutor.java tajo-core/src/main/proto/ResourceProtos.proto

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development