Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1282

Cleanup the relationship of QueryInProgress and QueryJobManager

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: TajoMaster
    • Labels:
      None

      Description

      Each QueryInProgress instance maintains an individual event handler involving each thread. It complicates the relationship between QueryJobManager and QueryInprogress. The main objective of this issue is to remove each event handler from QueryInProgress and distinguishes their roles of both classes.

      1. TAJO-1282_2.patch
        13 kB
        Hyunsik Choi
      2. TAJO-1282.patch
        4 kB
        Hyunsik Choi

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #182 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/182/)
          TAJO-1282: Cleanup the relationship of QueryInProgress and QueryJobManager. (hyunsik: rev 307c6c9f558ae96ffa2f7351ffa3dc5788ba4dbb)

          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryInProgress.java
          • tajo-core/src/main/java/org/apache/tajo/master/QueryJobManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
          • CHANGES
          • tajo-core/src/test/java/org/apache/tajo/querymaster/TestKillQuery.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryJobEvent.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterService.java
          • tajo-core/src/main/proto/TajoMasterProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java
          • tajo-core/src/main/java/org/apache/tajo/master/scheduler/SimpleFifoScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #182 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/182/ ) TAJO-1282 : Cleanup the relationship of QueryInProgress and QueryJobManager. (hyunsik: rev 307c6c9f558ae96ffa2f7351ffa3dc5788ba4dbb) tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryInProgress.java tajo-core/src/main/java/org/apache/tajo/master/QueryJobManager.java tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java CHANGES tajo-core/src/test/java/org/apache/tajo/querymaster/TestKillQuery.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryJobEvent.java tajo-core/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterService.java tajo-core/src/main/proto/TajoMasterProtocol.proto tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java tajo-core/src/main/java/org/apache/tajo/master/scheduler/SimpleFifoScheduler.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #543 (See https://builds.apache.org/job/Tajo-master-build/543/)
          TAJO-1282: Cleanup the relationship of QueryInProgress and QueryJobManager. (hyunsik: rev 307c6c9f558ae96ffa2f7351ffa3dc5788ba4dbb)

          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterService.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryInProgress.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/scheduler/SimpleFifoScheduler.java
          • CHANGES
          • tajo-core/src/test/java/org/apache/tajo/querymaster/TestKillQuery.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
          • tajo-core/src/main/proto/TajoMasterProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryJobEvent.java
          • tajo-core/src/main/java/org/apache/tajo/master/QueryJobManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #543 (See https://builds.apache.org/job/Tajo-master-build/543/ ) TAJO-1282 : Cleanup the relationship of QueryInProgress and QueryJobManager. (hyunsik: rev 307c6c9f558ae96ffa2f7351ffa3dc5788ba4dbb) tajo-core/src/main/java/org/apache/tajo/master/TajoMasterService.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMasterTask.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryInProgress.java tajo-core/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java tajo-core/src/main/java/org/apache/tajo/master/scheduler/SimpleFifoScheduler.java CHANGES tajo-core/src/test/java/org/apache/tajo/querymaster/TestKillQuery.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/proto/TajoMasterProtocol.proto tajo-core/src/main/java/org/apache/tajo/querymaster/QueryJobEvent.java tajo-core/src/main/java/org/apache/tajo/master/QueryJobManager.java tajo-core/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/334

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/334
          Hide
          hyunsik Hyunsik Choi added a comment -

          committed. Thanks!

          Show
          hyunsik Hyunsik Choi added a comment - committed. Thanks!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on the pull request:

          https://github.com/apache/tajo/pull/334#issuecomment-69295584

          +1 LGTM!
          Ship it!

          Show
          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/334#issuecomment-69295584 +1 LGTM! Ship it!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on the pull request:

          https://github.com/apache/tajo/pull/334#issuecomment-69287398

          Could you remove isQueryMasterStopped in QueryInProgress.stop() ?
          It seems unnecessary to wait.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/334#issuecomment-69287398 Could you remove isQueryMasterStopped in QueryInProgress.stop() ? It seems unnecessary to wait.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/334#issuecomment-69222736

          I rebased and cleaned up the relationship. This patch does as follows:

          • Remove CompisiteService from QueryInProgress
          • Remove AsyncDispatcher from QueryInProgress
          • Change QueryJobManager to handle all query-related events and routes them to each QueryInProgress instance.
          • Remove the following event types in QueryJobEvent:
          • QUERY_JOB_START
          • QUERY_JOB_FINISH
          • QUERY_MASTER_START
          • QUERY_MASTER_STOP
          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/334#issuecomment-69222736 I rebased and cleaned up the relationship. This patch does as follows: Remove CompisiteService from QueryInProgress Remove AsyncDispatcher from QueryInProgress Change QueryJobManager to handle all query-related events and routes them to each QueryInProgress instance. Remove the following event types in QueryJobEvent: QUERY_JOB_START QUERY_JOB_FINISH QUERY_MASTER_START QUERY_MASTER_STOP
          Hide
          hyunsik Hyunsik Choi added a comment -

          It's an initial work. There still remain works.

          Show
          hyunsik Hyunsik Choi added a comment - It's an initial work. There still remain works.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user hyunsik opened a pull request:

          https://github.com/apache/tajo/pull/334

          TAJO-1282: Cleanup the relationship of QueryInProgress and QueryJobManager.

          It's initial work.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hyunsik/tajo TAJO-1282

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/334.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #334


          commit 57e561bc6789b395bda8848583593b3d0f20ec62
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-01-06T17:03:33Z

          Remove TajoMasterService::stopQueryMaster API call.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/334 TAJO-1282 : Cleanup the relationship of QueryInProgress and QueryJobManager. It's initial work. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1282 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/334.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #334 commit 57e561bc6789b395bda8848583593b3d0f20ec62 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-01-06T17:03:33Z Remove TajoMasterService::stopQueryMaster API call.

            People

            • Assignee:
              hyunsik Hyunsik Choi
              Reporter:
              hyunsik Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development