Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1228

TajoClient should communicate with only TajoMaster without TajoWorker

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: Java Client
    • Labels:
      None

      Description

      Currently, TajoClient connects with TajoMaster as well as TajoWorker. A query submission is passed from TajoClient to TajoMaster, and than TajoMaster forwards the query to one query master running on a TajoWorker. After than, a client monitors the query progress through the query master.

      In other words, TajoClient contacts both TajoMaster and TajoWorker. It has three disadvantages:

      • a network firewall should allow connections to TajoWorker nodes.
      • all components should have complex states and communication logic one another.
      • TAJO-1160 is trying to remove hadoop dependency from TajoClient. In TAJO-1160, TajoClient will use only some server which forwards query results to itself instead of reading directly HDFS. TajoMaster would be the best component to forward the query results because TajoClient always connects with TajoMaster. So, if TajoClient communicates with only TajoMaster, all client logic and protocol would be simple.
      1. TAJO-1228.patch
        34 kB
        Hyunsik Choi
      2. TAJO-1228_4.patch
        35 kB
        Hyunsik Choi
      3. TAJO-1228_3.patch
        33 kB
        Hyunsik Choi
      4. TAJO-1228_2.patch
        52 kB
        Hyunsik Choi

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Tajo-master-CODEGEN-build #172 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/172/)
          TAJO-1228: TajoClient should communicate with only TajoMaster without TajoWorker. (hyunsik: rev cb9793b990f1c882e3371a44e6c3f28fe913c1a7)

          • tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-core/src/main/proto/TajoMasterProtocol.proto
          • tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
          • CHANGES
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
          • tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
          • tajo-client/src/main/proto/QueryMasterClientProtocol.proto
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #172 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/172/ ) TAJO-1228 : TajoClient should communicate with only TajoMaster without TajoWorker. (hyunsik: rev cb9793b990f1c882e3371a44e6c3f28fe913c1a7) tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-client/src/main/proto/ClientProtos.proto tajo-core/src/main/proto/TajoMasterProtocol.proto tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java CHANGES tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java tajo-client/src/main/proto/QueryMasterClientProtocol.proto
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #533 (See https://builds.apache.org/job/Tajo-master-build/533/)
          TAJO-1228: TajoClient should communicate with only TajoMaster without TajoWorker. (hyunsik: rev cb9793b990f1c882e3371a44e6c3f28fe913c1a7)

          • tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-core/src/main/proto/TajoMasterProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          • CHANGES
          • tajo-client/src/main/proto/QueryMasterClientProtocol.proto
          • tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #533 (See https://builds.apache.org/job/Tajo-master-build/533/ ) TAJO-1228 : TajoClient should communicate with only TajoMaster without TajoWorker. (hyunsik: rev cb9793b990f1c882e3371a44e6c3f28fe913c1a7) tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/src/main/proto/TajoMasterProtocol.proto tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java tajo-client/src/main/proto/ClientProtos.proto tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java CHANGES tajo-client/src/main/proto/QueryMasterClientProtocol.proto tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          Hide
          hyunsik Hyunsik Choi added a comment -

          committed to master branch. Thank you for the review.

          Show
          hyunsik Hyunsik Choi added a comment - committed to master branch. Thank you for the review.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/317

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/317
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/317#issuecomment-68678042

          +1

          I tested the patch on tpc-h cluster. Queries ran successfully and codes look good to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/317#issuecomment-68678042 +1 I tested the patch on tpc-h cluster. Queries ran successfully and codes look good to me.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12689424/TAJO-1228_4.patch
          against master revision release-0.9.0-rc0-112-gfd49bff.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 176 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 299 release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-client tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/563//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/563//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12689424/TAJO-1228_4.patch against master revision release-0.9.0-rc0-112-gfd49bff. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 176 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 299 release audit warnings. +1 core tests. The patch passed unit tests in tajo-client tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/563//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/563//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/563//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/317#issuecomment-68147410

          Hi @hyunsik

          If possible, could you check the patch on testing cluster?
          I failed Q3 query on tpc-h benchmark dataset with the patch as follows:

          2014-12-27 01:18:08,273 ERROR org.apache.tajo.catalog.CatalogServer: java.lang.NullPointerException
          2014-12-27 01:18:08,273 ERROR org.apache.tajo.catalog.AbstractCatalogClient: com.google.protobuf.ServiceException: java.lang.NullPointerException
          com.google.protobuf.ServiceException: com.google.protobuf.ServiceException: java.lang.NullPointerException
          at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
          at org.apache.tajo.catalog.AbstractCatalogClient.getTableDesc(AbstractCatalogClient.java:243)
          at org.apache.tajo.catalog.AbstractCatalogClient.getTableDesc(AbstractCatalogClient.java:262)
          at org.apache.tajo.plan.LogicalPlanPreprocessor.visitRelation(LogicalPlanPreprocessor.java:385)
          at org.apache.tajo.plan.LogicalPlanPreprocessor.visitRelation(LogicalPlanPreprocessor.java:44)
          at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:98)
          at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visitRelationList(BaseAlgebraVisitor.java:418)
          at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:95)
          at org.apache.tajo.plan.LogicalPlanPreprocessor.visitProjection(LogicalPlanPreprocessor.java:197)
          at org.apache.tajo.plan.LogicalPlanPreprocessor.visitProjection(LogicalPlanPreprocessor.java:44)
          at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:59)
          at org.apache.tajo.plan.LogicalPlanner.createPlan(LogicalPlanner.java:141)
          at org.apache.tajo.plan.LogicalPlanner.createPlan(LogicalPlanner.java:131)
          at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:232)
          at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:161)
          at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.submitQuery(TajoMasterClientService.java:284)
          at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:543)
          at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/317#issuecomment-68147410 Hi @hyunsik If possible, could you check the patch on testing cluster? I failed Q3 query on tpc-h benchmark dataset with the patch as follows: 2014-12-27 01:18:08,273 ERROR org.apache.tajo.catalog.CatalogServer: java.lang.NullPointerException 2014-12-27 01:18:08,273 ERROR org.apache.tajo.catalog.AbstractCatalogClient: com.google.protobuf.ServiceException: java.lang.NullPointerException com.google.protobuf.ServiceException: com.google.protobuf.ServiceException: java.lang.NullPointerException at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105) at org.apache.tajo.catalog.AbstractCatalogClient.getTableDesc(AbstractCatalogClient.java:243) at org.apache.tajo.catalog.AbstractCatalogClient.getTableDesc(AbstractCatalogClient.java:262) at org.apache.tajo.plan.LogicalPlanPreprocessor.visitRelation(LogicalPlanPreprocessor.java:385) at org.apache.tajo.plan.LogicalPlanPreprocessor.visitRelation(LogicalPlanPreprocessor.java:44) at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:98) at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visitRelationList(BaseAlgebraVisitor.java:418) at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:95) at org.apache.tajo.plan.LogicalPlanPreprocessor.visitProjection(LogicalPlanPreprocessor.java:197) at org.apache.tajo.plan.LogicalPlanPreprocessor.visitProjection(LogicalPlanPreprocessor.java:44) at org.apache.tajo.plan.algebra.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:59) at org.apache.tajo.plan.LogicalPlanner.createPlan(LogicalPlanner.java:141) at org.apache.tajo.plan.LogicalPlanner.createPlan(LogicalPlanner.java:131) at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:232) at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:161) at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.submitQuery(TajoMasterClientService.java:284) at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:543) at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12689181/TAJO-1228_3.patch
          against master revision release-0.9.0-rc0-112-gfd49bff.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 176 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 625 release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-client tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/560//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/560//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12689181/TAJO-1228_3.patch against master revision release-0.9.0-rc0-112-gfd49bff. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 176 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 625 release audit warnings. +1 core tests. The patch passed unit tests in tajo-client tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/560//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/560//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/560//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/317#issuecomment-68144562

          Thanks @hyunsik, I merged the patch successfully. I'll start to review it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/317#issuecomment-68144562 Thanks @hyunsik, I merged the patch successfully. I'll start to review it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/317#issuecomment-68144407

          I've rebased it. Thanks1

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/317#issuecomment-68144407 I've rebased it. Thanks1
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/317#issuecomment-68144062

          Hi @hyunsik

          Could you rebase sources against the apache master branch?
          I can't merge as follows:
          Auto-merging tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          CONFLICT (content): Merge conflict in tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          Auto-merging tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          Automatic merge failed; fix conflicts and then commit the result.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/317#issuecomment-68144062 Hi @hyunsik Could you rebase sources against the apache master branch? I can't merge as follows: Auto-merging tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java CONFLICT (content): Merge conflict in tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java Auto-merging tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java Automatic merge failed; fix conflicts and then commit the result.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12689037/TAJO-1228_2.patch
          against master revision release-0.9.0-rc0-109-gdb54965.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 251 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 624 release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-client tajo-common tajo-core tajo-plan.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/559//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/559//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12689037/TAJO-1228_2.patch against master revision release-0.9.0-rc0-109-gdb54965. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 251 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 624 release audit warnings. +1 core tests. The patch passed unit tests in tajo-client tajo-common tajo-core tajo-plan. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/559//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/559//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/559//console This message is automatically generated.
          Hide
          hyunsik Hyunsik Choi added a comment -

          I trigger the jenkins test.

          Show
          hyunsik Hyunsik Choi added a comment - I trigger the jenkins test.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch
          against master revision release-0.9.0-rc0-108-g09cad22.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 586 release audit warnings.

          -1 core tests. The patch failed these unit tests in tajo-client tajo-core:
          org.apache.tajo.client.TestTajoClient

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/558//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/558//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch against master revision release-0.9.0-rc0-108-g09cad22. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 586 release audit warnings. -1 core tests. The patch failed these unit tests in tajo-client tajo-core: org.apache.tajo.client.TestTajoClient Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/558//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/558//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/558//console This message is automatically generated.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch
          against master revision release-0.9.0-rc0-108-g09cad22.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 122 release audit warnings.

          -1 core tests. The patch failed these unit tests in tajo-client tajo-core:
          org.apache.tajo.engine.query.TestJoinQuery

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/557//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/557//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch against master revision release-0.9.0-rc0-108-g09cad22. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 122 release audit warnings. -1 core tests. The patch failed these unit tests in tajo-client tajo-core: org.apache.tajo.engine.query.TestJoinQuery Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/557//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/557//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/557//console This message is automatically generated.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch
          against master revision release-0.9.0-rc0-108-g09cad22.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 586 release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-client tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/555//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/555//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12688989/TAJO-1228.patch against master revision release-0.9.0-rc0-108-g09cad22. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 175 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 586 release audit warnings. +1 core tests. The patch passed unit tests in tajo-client tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/555//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/555//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/555//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user hyunsik opened a pull request:

          https://github.com/apache/tajo/pull/317

          TAJO-1228: TajoClient should communicate with only TajoMaster without TajoWorker.

          See the description at https://issues.apache.org/jira/browse/TAJO-1228.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hyunsik/tajo TAJO-1228

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/317.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #317


          commit 64fb64ea9b3de3681a22c9f64e90beff06d001e1
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-21T11:23:17Z

          Initial work

          commit 0241bbcd86517604cc997471860b1f5af4a2f4b5
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-21T12:19:24Z

          All test passed after getStatus from TajoMaster.

          commit 01a1aed280b0007eb2d07b6548660e7eba18d12a
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-21T15:37:37Z

          Completely remove connection of query master of query status.

          commit a8983b73911f8c25cac270f04137101d2cb2930d
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-24T02:16:25Z

          Passed all unit tests and removed worker client service.

          commit 546837ee8176ebecc38f14050fdbc0394ea0fd43
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-24T03:15:37Z

          Remove queryMasterMap from SessionConnection.

          commit e158fe72fecb111fbb7b72f8c76397901b6762a2
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-24T03:47:34Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1228

          Conflicts:
          tajo-core/src/main/java/org/apache/tajo/master/GlobalEngine.java
          tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java

          commit 9de88de5da0a27145ae3626ed2c9db47b0d8092d
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2014-12-24T04:03:59Z

          Clean up codes.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/317 TAJO-1228 : TajoClient should communicate with only TajoMaster without TajoWorker. See the description at https://issues.apache.org/jira/browse/TAJO-1228 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1228 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/317.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #317 commit 64fb64ea9b3de3681a22c9f64e90beff06d001e1 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-21T11:23:17Z Initial work commit 0241bbcd86517604cc997471860b1f5af4a2f4b5 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-21T12:19:24Z All test passed after getStatus from TajoMaster. commit 01a1aed280b0007eb2d07b6548660e7eba18d12a Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-21T15:37:37Z Completely remove connection of query master of query status. commit a8983b73911f8c25cac270f04137101d2cb2930d Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-24T02:16:25Z Passed all unit tests and removed worker client service. commit 546837ee8176ebecc38f14050fdbc0394ea0fd43 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-24T03:15:37Z Remove queryMasterMap from SessionConnection. commit e158fe72fecb111fbb7b72f8c76397901b6762a2 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-24T03:47:34Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1228 Conflicts: tajo-core/src/main/java/org/apache/tajo/master/GlobalEngine.java tajo-core/src/test/java/org/apache/tajo/TajoTestingCluster.java commit 9de88de5da0a27145ae3626ed2c9db47b0d8092d Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-12-24T04:03:59Z Clean up codes.
          Hide
          hyunsik Hyunsik Choi added a comment -

          I'll change QueryInProgress and QueryInfo as follows:

          • Add TableDesc to TajoHeartBeat message.
          • buildTajoHeartBeat sets TableDesc of result set TajoHeartBeat if a query is succeeded
          • If the query is one of terminal state and heartbeat message includes TableDesc, QueryInProgress::heartbeat sets the result to QueryInfo.

          Also, I'll change QueryClientImpl to get a running query status and a query result from TajoMaster.

          Show
          hyunsik Hyunsik Choi added a comment - I'll change QueryInProgress and QueryInfo as follows: Add TableDesc to TajoHeartBeat message. buildTajoHeartBeat sets TableDesc of result set TajoHeartBeat if a query is succeeded If the query is one of terminal state and heartbeat message includes TableDesc, QueryInProgress::heartbeat sets the result to QueryInfo. Also, I'll change QueryClientImpl to get a running query status and a query result from TajoMaster.

            People

            • Assignee:
              hyunsik Hyunsik Choi
              Reporter:
              hyunsik Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development