Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1311

Enable Scattered Hash Shuffle for CTAS statement.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: distributed query plan
    • Labels:
      None

      Description

      Currently, Tajo just use hash shuffle for CTAS statement. If you use large partitioned table for target table, you will see poor performance of wrong task number. Thus, we need to calculate the number for the partitioned table with scattered hash shuffle.

      1. TAJO-1311.patch
        39 kB
        Jaehwa Jung
      2. TAJO-1311_2.patch
        39 kB
        Jaehwa Jung
      3. TAJO-1311_3.patch
        39 kB
        Jaehwa Jung

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user blrunner opened a pull request:

        https://github.com/apache/tajo/pull/357

        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement.

        Currently, Tajo just use hash shuffle for CTAS statement. If you use large partitioned table for target table, you will see poor performance of wrong task number. Thus, we need to calculate the number for the partitioned table with scattered hash shuffle.

        For reference, CTAS includes two bugs as follows:

        • Tajo doesn't verify target column number for CTAS statement
        • If target table and source table are column partitioned table, Tajo doesn't support CTAS statement.

        I'll create another jira issues at once.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/blrunner/tajo TAJO-1311

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/357.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #357


        commit 6eb2d10cb1d7805d46436829f443c92ef0d15f29
        Author: JaeHwa Jung <blrunner@apache.org>
        Date: 2015-01-22T14:37:34Z

        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement.

        commit 6e5e14e82d21d81de1ceaedf53307acc39ccf7c0
        Author: JaeHwa Jung <blrunner@apache.org>
        Date: 2015-01-22T14:37:58Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1311


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user blrunner opened a pull request: https://github.com/apache/tajo/pull/357 TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement. Currently, Tajo just use hash shuffle for CTAS statement. If you use large partitioned table for target table, you will see poor performance of wrong task number. Thus, we need to calculate the number for the partitioned table with scattered hash shuffle. For reference, CTAS includes two bugs as follows: Tajo doesn't verify target column number for CTAS statement If target table and source table are column partitioned table, Tajo doesn't support CTAS statement. I'll create another jira issues at once. You can merge this pull request into a Git repository by running: $ git pull https://github.com/blrunner/tajo TAJO-1311 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #357 commit 6eb2d10cb1d7805d46436829f443c92ef0d15f29 Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-01-22T14:37:34Z TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement. commit 6e5e14e82d21d81de1ceaedf53307acc39ccf7c0 Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-01-22T14:37:58Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1311
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/357#discussion_r23589661

        — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java —
        @@ -854,6 +855,14 @@ private PhysicalExec createSortBasedColumnPartitionStorePlan(TaskAttemptContext
        sortSpecs[i++] = new SortSpec(insertNode.getProjectedSchema().getColumn(id), true, false);
        }
        }
        + } else if (storeTableNode.getType() == NodeType.CREATE_TABLE) {
        + int i = 0;
        + for (int j = 0; j < partitionKeyColumns.length; j++) {
        + int id = storeTableNode.getOutSchema().getColumns().size() + j;
        — End diff –

        In my opinion, we should consider the qualified name for CTAS with defined column.
        @blrunner @hyunsik , How do you think?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on a diff in the pull request: https://github.com/apache/tajo/pull/357#discussion_r23589661 — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java — @@ -854,6 +855,14 @@ private PhysicalExec createSortBasedColumnPartitionStorePlan(TaskAttemptContext sortSpecs [i++] = new SortSpec(insertNode.getProjectedSchema().getColumn(id), true, false); } } + } else if (storeTableNode.getType() == NodeType.CREATE_TABLE) { + int i = 0; + for (int j = 0; j < partitionKeyColumns.length; j++) { + int id = storeTableNode.getOutSchema().getColumns().size() + j; — End diff – In my opinion, we should consider the qualified name for CTAS with defined column. @blrunner @hyunsik , How do you think?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/357#discussion_r24139386

        — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java —
        @@ -1130,6 +1130,18 @@ private void setShuffleKeysFromPartitionedTableStore(StoreTableNode node, DataCh
        }
        channel.setShuffleKeys(shuffleKeys);
        channel.setShuffleType(SCATTERED_HASH_SHUFFLE);
        + } else if (node.getType() == NodeType.CREATE_TABLE) {
        — End diff –

        The newly added code is the mostly same as those of INSERT. It would be nice if some code to get ProjectSchema and InSchema are distinguished.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/357#discussion_r24139386 — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java — @@ -1130,6 +1130,18 @@ private void setShuffleKeysFromPartitionedTableStore(StoreTableNode node, DataCh } channel.setShuffleKeys(shuffleKeys); channel.setShuffleType(SCATTERED_HASH_SHUFFLE); + } else if (node.getType() == NodeType.CREATE_TABLE) { — End diff – The newly added code is the mostly same as those of INSERT. It would be nice if some code to get ProjectSchema and InSchema are distinguished.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/357#discussion_r24139933

        — Diff: tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java —
        @@ -45,54 +45,72 @@
        import org.apache.tajo.util.KeyValueSet;
        import org.apache.tajo.worker.TajoWorker;
        import org.junit.Test;
        +import org.junit.runner.RunWith;
        +import org.junit.runners.Parameterized;

        import java.io.IOException;
        import java.sql.ResultSet;
        -import java.util.ArrayList;
        -import java.util.List;
        -import java.util.Map;
        -import java.util.Random;
        +import java.util.*;

        import static org.apache.tajo.TajoConstants.DEFAULT_DATABASE_NAME;
        import static org.apache.tajo.plan.serder.PlanProto.ShuffleType.SCATTERED_HASH_SHUFFLE;
        import static org.junit.Assert.*;

        +@RunWith(Parameterized.class)
        public class TestTablePartitions extends QueryTestCaseBase {

        + private NodeType nodeType;

        • public TestTablePartitions() throws IOException {
          + public TestTablePartitions(NodeType nodeType) throws IOException { super(TajoConstants.DEFAULT_DATABASE_NAME); + this.nodeType = nodeType; + }

          +
          + @Parameterized.Parameters
          + public static Collection<Object[]> generateParameters() {
          + return Arrays.asList(new Object[][]

          Unknown macro: { + //type + {NodeType.INSERT}, + {NodeType.CREATE_TABLE}, + }

          );
          }

        @Test
        public final void testCreateColumnPartitionedTable() throws Exception {
        + ResultSet res = null;
        String tableName = CatalogUtil.normalizeIdentifier("testCreateColumnPartitionedTable");

        • ResultSet res = executeString(
          +
          + if (nodeType == NodeType.INSERT) {
          + res = executeString(
          "create table " + tableName + " (col1 int4, col2 int4) partition by column(key float8) ");
        • res.close();
          + res.close();
        • assertTrue(catalog.existsTable(DEFAULT_DATABASE_NAME, tableName));
        • assertEquals(2, catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName).getSchema().size());
        • assertEquals(3, catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName).getLogicalSchema().size());
          + assertTrue(catalog.existsTable(DEFAULT_DATABASE_NAME, tableName));
            • End diff –

        The assert statements for CREATE or INSERT are duplicated. This kind of assert statements in other unit test methods are also duplicated. You need to keep only one.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/357#discussion_r24139933 — Diff: tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java — @@ -45,54 +45,72 @@ import org.apache.tajo.util.KeyValueSet; import org.apache.tajo.worker.TajoWorker; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; import java.io.IOException; import java.sql.ResultSet; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.Random; +import java.util.*; import static org.apache.tajo.TajoConstants.DEFAULT_DATABASE_NAME; import static org.apache.tajo.plan.serder.PlanProto.ShuffleType.SCATTERED_HASH_SHUFFLE; import static org.junit.Assert.*; +@RunWith(Parameterized.class) public class TestTablePartitions extends QueryTestCaseBase { + private NodeType nodeType; public TestTablePartitions() throws IOException { + public TestTablePartitions(NodeType nodeType) throws IOException { super(TajoConstants.DEFAULT_DATABASE_NAME); + this.nodeType = nodeType; + } + + @Parameterized.Parameters + public static Collection<Object[]> generateParameters() { + return Arrays.asList(new Object[][] Unknown macro: { + //type + {NodeType.INSERT}, + {NodeType.CREATE_TABLE}, + } ); } @Test public final void testCreateColumnPartitionedTable() throws Exception { + ResultSet res = null; String tableName = CatalogUtil.normalizeIdentifier("testCreateColumnPartitionedTable"); ResultSet res = executeString( + + if (nodeType == NodeType.INSERT) { + res = executeString( "create table " + tableName + " (col1 int4, col2 int4) partition by column(key float8) "); res.close(); + res.close(); assertTrue(catalog.existsTable(DEFAULT_DATABASE_NAME, tableName)); assertEquals(2, catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName).getSchema().size()); assertEquals(3, catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName).getLogicalSchema().size()); + assertTrue(catalog.existsTable(DEFAULT_DATABASE_NAME, tableName)); End diff – The assert statements for CREATE or INSERT are duplicated. This kind of assert statements in other unit test methods are also duplicated. You need to keep only one.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/357#discussion_r24142323

        — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java —
        @@ -854,6 +855,14 @@ private PhysicalExec createSortBasedColumnPartitionStorePlan(TaskAttemptContext
        sortSpecs[i++] = new SortSpec(insertNode.getProjectedSchema().getColumn(id), true, false);
        }
        }
        + } else if (storeTableNode.getType() == NodeType.CREATE_TABLE) {
        + int i = 0;
        + for (int j = 0; j < partitionKeyColumns.length; j++) {
        + int id = storeTableNode.getOutSchema().getColumns().size() + j;
        — End diff –

        In 'CREATE TABLE', it is to hard to use qualified names of partition keys.

        CreateTableNode's in/out schema only contains physically stored columns and does not contain logical columns (i.e., partition key columns). The way described in the patch will work correctly.

        But, since this way is not intuitive, we need to refactor CreateTableNode and InsertTableNode to provide a better way to access physical columns corresponding to partition keys.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/357#discussion_r24142323 — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java — @@ -854,6 +855,14 @@ private PhysicalExec createSortBasedColumnPartitionStorePlan(TaskAttemptContext sortSpecs [i++] = new SortSpec(insertNode.getProjectedSchema().getColumn(id), true, false); } } + } else if (storeTableNode.getType() == NodeType.CREATE_TABLE) { + int i = 0; + for (int j = 0; j < partitionKeyColumns.length; j++) { + int id = storeTableNode.getOutSchema().getColumns().size() + j; — End diff – In 'CREATE TABLE', it is to hard to use qualified names of partition keys. CreateTableNode's in/out schema only contains physically stored columns and does not contain logical columns (i.e., partition key columns). The way described in the patch will work correctly. But, since this way is not intuitive, we need to refactor CreateTableNode and InsertTableNode to provide a better way to access physical columns corresponding to partition keys.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/357#issuecomment-75350796

        Any progress?

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/357#issuecomment-75350796 Any progress?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/357#issuecomment-78222253

        Sorry guys, I have been very busy.
        I'll restart it shortly.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/357#issuecomment-78222253 Sorry guys, I have been very busy. I'll restart it shortly.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/357#issuecomment-84643681

        @hyunsik

        I'm sorry to give the feedback lately.
        I cleaned up duplicated codes.
        Could you check the patch again?

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/357#issuecomment-84643681 @hyunsik I'm sorry to give the feedback lately. I cleaned up duplicated codes. Could you check the patch again?
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12706399/TAJO-1311.patch
        against master revision release-0.9.0-rc0-213-g3aaff38.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-core.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/637//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/637//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/637//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12706399/TAJO-1311.patch against master revision release-0.9.0-rc0-213-g3aaff38. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/637//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/637//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/637//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/357#issuecomment-87514110

        It seems as if there are some problems on commit logs.
        I'll create new pull request soon.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/357#issuecomment-87514110 It seems as if there are some problems on commit logs. I'll create new pull request soon.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner closed the pull request at:

        https://github.com/apache/tajo/pull/357

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner closed the pull request at: https://github.com/apache/tajo/pull/357
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user blrunner opened a pull request:

        https://github.com/apache/tajo/pull/478

        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement.

        Currently, Tajo just use hash shuffle for CTAS statement. If you use large partitioned table for target table, you will see poor performance of wrong task number. Thus, we need to calculate the number for the partitioned table with scattered hash shuffle.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/blrunner/tajo TAJO-1311-NEW

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/478.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #478


        commit 8a7310e34bfde95371a6f5df89154fa1150c029a
        Author: JaeHwa Jung <blrunner@apache.org>
        Date: 2015-03-30T02:35:40Z

        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user blrunner opened a pull request: https://github.com/apache/tajo/pull/478 TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement. Currently, Tajo just use hash shuffle for CTAS statement. If you use large partitioned table for target table, you will see poor performance of wrong task number. Thus, we need to calculate the number for the partitioned table with scattered hash shuffle. You can merge this pull request into a Git repository by running: $ git pull https://github.com/blrunner/tajo TAJO-1311 -NEW Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/478.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #478 commit 8a7310e34bfde95371a6f5df89154fa1150c029a Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-03-30T02:35:40Z TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement.
        Hide
        tajoqa Tajo QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708074/TAJO-1311_2.patch
        against master revision release-0.9.0-rc0-225-g7f77593.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-core.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/674//testReport/
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/674//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708074/TAJO-1311_2.patch against master revision release-0.9.0-rc0-225-g7f77593. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/674//testReport/ Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/674//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/478#issuecomment-89678823

        Rebasing commits against the master branch.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/478#issuecomment-89678823 Rebasing commits against the master branch.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12709462/TAJO-1311_3.patch
        against master revision release-0.9.0-rc0-237-gb0abff8.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-core.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/703//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/703//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/703//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12709462/TAJO-1311_3.patch against master revision release-0.9.0-rc0-237-gb0abff8. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/703//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/703//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/703//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/478#issuecomment-89763873

        Testcases in travis-ci failed, but testcases in apache build servers has been passed. Please see details in the following link.
        https://builds.apache.org/job/PreCommit-TAJO-Build/703//testReport/

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/478#issuecomment-89763873 Testcases in travis-ci failed, but testcases in apache build servers has been passed. Please see details in the following link. https://builds.apache.org/job/PreCommit-TAJO-Build/703//testReport/
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/478#issuecomment-93916610

        Rebased

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/478#issuecomment-93916610 Rebased
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/478#issuecomment-96580434

        +1 thanks for your contribution!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/478#issuecomment-96580434 +1 thanks for your contribution!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/478

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/478
        Hide
        blrunner Jaehwa Jung added a comment -

        Thanks Jihoon Son
        I've just committed this to the master branch.

        Show
        blrunner Jaehwa Jung added a comment - Thanks Jihoon Son I've just committed this to the master branch.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #326 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/326/)
        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement. (jaehwa) (blrunner: rev b50831ffa66d94b67a419a95f81beed8fcab8ba2)

        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • CHANGES
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #326 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/326/ ) TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement. (jaehwa) (blrunner: rev b50831ffa66d94b67a419a95f81beed8fcab8ba2) tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java CHANGES
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #688 (See https://builds.apache.org/job/Tajo-master-build/688/)
        TAJO-1311: Enable Scattered Hash Shuffle for CTAS statement. (jaehwa) (blrunner: rev b50831ffa66d94b67a419a95f81beed8fcab8ba2)

        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #688 (See https://builds.apache.org/job/Tajo-master-build/688/ ) TAJO-1311 : Enable Scattered Hash Shuffle for CTAS statement. (jaehwa) (blrunner: rev b50831ffa66d94b67a419a95f81beed8fcab8ba2) tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java CHANGES tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java

          People

          • Assignee:
            blrunner Jaehwa Jung
            Reporter:
            blrunner Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development