Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: Catalog, SQL Parser
    • Labels:
      None

      Description

      Currently, users can use arbitrary partition path using alter table statement. But they can't scan their partitions because PartitionedTableRewriter just build tuple from partition path. Fundamentally, we should allow arbitary partition path. But we need to avoid an exception by above situation until tajo allow arbitrary partition path. So it seems that alter table add partition statement is disabled temporarily.

      1. TAJO-1887.patch
        48 kB
        Jaehwa Jung

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-0.11.0-build #80 (See https://builds.apache.org/job/Tajo-0.11.0-build/80/)
          TAJO-1887: Disable the alter table add partition statement temporarily. (blrunner: rev 3f33a6b14139ecb83e19a6069c27cb867dd6a95f)

          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java
          • tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java
          • tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java
          • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-docs/src/main/sphinx/sql_language/alter_table.rst
          • tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java
          • tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result
          • tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java
          • tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • CHANGES
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-0.11.0-build #80 (See https://builds.apache.org/job/Tajo-0.11.0-build/80/ ) TAJO-1887 : Disable the alter table add partition statement temporarily. (blrunner: rev 3f33a6b14139ecb83e19a6069c27cb867dd6a95f) tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-docs/src/main/sphinx/sql_language/alter_table.rst tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java CHANGES tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #915 (See https://builds.apache.org/job/Tajo-master-build/915/)
          TAJO-1887: Disable the alter table add partition statement temporarily. (blrunner: rev 4564c37fa7a54888424134d75ec06adcbb8c10c0)

          • tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java
          • CHANGES
          • tajo-docs/src/main/sphinx/sql_language/alter_table.rst
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java
          • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
          • tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result
          • tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java
          • tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java
          • tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java
          • tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #915 (See https://builds.apache.org/job/Tajo-master-build/915/ ) TAJO-1887 : Disable the alter table add partition statement temporarily. (blrunner: rev 4564c37fa7a54888424134d75ec06adcbb8c10c0) tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java CHANGES tajo-docs/src/main/sphinx/sql_language/alter_table.rst tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #546 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/546/)
          TAJO-1887: Disable the alter table add partition statement temporarily. (blrunner: rev 4564c37fa7a54888424134d75ec06adcbb8c10c0)

          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          • tajo-docs/src/main/sphinx/sql_language/alter_table.rst
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java
          • tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java
          • CHANGES
          • tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java
          • tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java
          • tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java
          • tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #546 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/546/ ) TAJO-1887 : Disable the alter table add partition statement temporarily. (blrunner: rev 4564c37fa7a54888424134d75ec06adcbb8c10c0) tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java tajo-docs/src/main/sphinx/sql_language/alter_table.rst tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogExceptions.java tajo-cli/src/main/java/org/apache/tajo/cli/tools/TajoDump.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalog.java tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-jdbc/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java CHANGES tajo-common/src/main/java/org/apache/tajo/exception/ReturnStateUtil.java tajo-core-tests/src/test/resources/results/TestTajoDump/testPartitionsDump.result tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestCatalogAgainstCaseSensitivity.java tajo-catalog/tajo-catalog-drivers/tajo-hive/src/test/java/org/apache/tajo/catalog/store/TestHiveCatalogStore.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestTajoDump.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogService.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestAlterTable.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tsql/TestTajoCli.java tajo-core-tests/src/test/resources/results/TestTajoCli/testAddPartitionNotimplementedException.result
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/785

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/785
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143894595

          +1
          @blrunner thank you for your contribution. The patch looks good to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143894595 +1 @blrunner thank you for your contribution. The patch looks good to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143184165

          Rebased and updated the document for alter table statement.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143184165 Rebased and updated the document for alter table statement.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143169444

          Could you rebase it against the latest revision?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143169444 Could you rebase it against the latest revision?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143125811

          I added negative unit test cases.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143125811 I added negative unit test cases.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143120822

          Could you add negative tests for ``ALTER ADD PARTITION``?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143120822 Could you add negative tests for ``ALTER ADD PARTITION``?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143110524

          Thanks @hyunsik
          I've reflected your comments just now.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143110524 Thanks @hyunsik I've reflected your comments just now.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/785#discussion_r40396064

          — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java —
          @@ -354,4 +354,14 @@ public Expr visitInsert(Context context, Stack<Expr> stack, Insert expr) throws

          return expr;
          }
          +
          + // TODO: This should be removed at TAJO-1891
          + @Override
          + public Expr visitAlterTable(Context context, Stack<Expr> stack, AlterTable expr) throws TajoException {
          + if (expr.getAlterTableOpType() == AlterTableOpType.ADD_PARTITION) {
          + context.state.addVerification(new UnsupportedException("ADD PARTITION"));
          — End diff –

          It should be UnimplementedException. UnsupportedException means there is no plan to support this feature.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/785#discussion_r40396064 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java — @@ -354,4 +354,14 @@ public Expr visitInsert(Context context, Stack<Expr> stack, Insert expr) throws return expr; } + + // TODO: This should be removed at TAJO-1891 + @Override + public Expr visitAlterTable(Context context, Stack<Expr> stack, AlterTable expr) throws TajoException { + if (expr.getAlterTableOpType() == AlterTableOpType.ADD_PARTITION) { + context.state.addVerification(new UnsupportedException("ADD PARTITION")); — End diff – It should be UnimplementedException. UnsupportedException means there is no plan to support this feature.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143108120

          Thanks @hyunsik

          I reflected your comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143108120 Thanks @hyunsik I reflected your comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/785#issuecomment-143065838

          Rather than commenting out for disabling the feature, I'd like to recommend to add some verification rule.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/785#issuecomment-143065838 Rather than commenting out for disabling the feature, I'd like to recommend to add some verification rule.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user blrunner opened a pull request:

          https://github.com/apache/tajo/pull/785

          TAJO-1887: Disable the alter table add partition statement temporarily.

          Currently, users can use arbitary partition path using alter table statement. But they can't scan their partitions because PartitionedTableRewriter just build tuple from partition path. Fundamentally, we should allow arbitary partition path. But we need to avoid an exception by above situation until tajo allow arbitary partition path. So it seems that alter table add partition statement is disabled temporarily.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/blrunner/tajo TAJO-1887

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/785.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #785


          commit 5311779ef2c481ef4f815843c44c877df0106d90
          Author: JaeHwa Jung <blrunner@apache.org>
          Date: 2015-09-24T14:38:29Z

          TAJO-1887: Disable the alter table add partition statement temporarily.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user blrunner opened a pull request: https://github.com/apache/tajo/pull/785 TAJO-1887 : Disable the alter table add partition statement temporarily. Currently, users can use arbitary partition path using alter table statement. But they can't scan their partitions because PartitionedTableRewriter just build tuple from partition path. Fundamentally, we should allow arbitary partition path. But we need to avoid an exception by above situation until tajo allow arbitary partition path. So it seems that alter table add partition statement is disabled temporarily. You can merge this pull request into a Git repository by running: $ git pull https://github.com/blrunner/tajo TAJO-1887 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/785.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #785 commit 5311779ef2c481ef4f815843c44c877df0106d90 Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-09-24T14:38:29Z TAJO-1887 : Disable the alter table add partition statement temporarily.

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              blrunner Jaehwa Jung
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development