Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1727

Avoid to create external table using TableSpace

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: SQL Parser
    • Labels:
      None

      Description

      I failed to create external table on HBaseTableSpace as following:

      • HBase Shell
        hbase(main):002:0> desc 'blog'
        
        Table blog is ENABLED                                                                                                                                                                                                               
        blog                                                                                                                                                                                                                                
        COLUMN FAMILIES DESCRIPTION                                                                                                                                                                                                         
        {NAME => 'content', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536
        ', IN_MEMORY => 'false', BLOCKCACHE => 'true'}                                                                                                                                                                                      
        {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', 
        IN_MEMORY => 'false', BLOCKCACHE => 'true'}                                                                                                                                                                                         
        2 row(s) in 0.2380 seconds
        
      • TSQL
        CREATE EXTERNAL TABLE hbase_blog (rowkey text, author text, register_date text, title text)  TABLESPACE hbasecluster1 USING hbase WITH ('table'='blog', 'columns'=':key,info:author,info:date,content:title');
        
        ERROR: hdfs://localhost:9010/tajo/warehouse/default/hbase_blog does not exist
        
      • TajoMaster Log
        Stack Trace:
        java.io.IOException: ERROR: hdfs://localhost:9010/tajo/warehouse/default/hbase_blog does not exist
        	at org.apache.tajo.storage.FileTablespace.createTable(FileTablespace.java:674)
        	at org.apache.tajo.master.exec.DDLExecutor.createTable(DDLExecutor.java:364)
        	at org.apache.tajo.master.exec.DDLExecutor.createTable(DDLExecutor.java:291)
        	at org.apache.tajo.master.exec.DDLExecutor.execute(DDLExecutor.java:93)
        	at org.apache.tajo.master.exec.QueryExecutor.execute(QueryExecutor.java:113)
        
      1. TAJO-1727_2.patch
        3 kB
        Jaehwa Jung
      2. TAJO-1727_3.patch
        4 kB
        Jaehwa Jung
      3. TAJO-1727.patch
        3 kB
        Jaehwa Jung

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user blrunner opened a pull request:

        https://github.com/apache/tajo/pull/732

        TAJO-1727: Unable to create external table on HBaseTableSpace

        When creating external table, add setting tablespace.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/blrunner/tajo TAJO-1727

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/732.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #732


        commit 2747af99be91d1c9b5502fcee4cbab18f00ee460
        Author: JaeHwa Jung <blrunner@apache.org>
        Date: 2015-09-05T16:17:34Z

        TAJO-1727: Unable to create external table on HBaseTableSpace


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user blrunner opened a pull request: https://github.com/apache/tajo/pull/732 TAJO-1727 : Unable to create external table on HBaseTableSpace When creating external table, add setting tablespace. You can merge this pull request into a Git repository by running: $ git pull https://github.com/blrunner/tajo TAJO-1727 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/732.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #732 commit 2747af99be91d1c9b5502fcee4cbab18f00ee460 Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-09-05T16:17:34Z TAJO-1727 : Unable to create external table on HBaseTableSpace
        Hide
        blrunner Jaehwa Jung added a comment -

        When creating external table, add setting tablespace.

        Show
        blrunner Jaehwa Jung added a comment - When creating external table, add setting tablespace.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12754331/TAJO-1727.patch
        against master revision release-0.9.0-rc0-445-gfd4a3f8.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-core tajo-core-tests:
        org.apache.tajo.engine.query.TestHBaseTable

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/854//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/854//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/854//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12754331/TAJO-1727.patch against master revision release-0.9.0-rc0-445-gfd4a3f8. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-core tajo-core-tests: org.apache.tajo.engine.query.TestHBaseTable Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/854//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/854//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/854//console This message is automatically generated.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12754372/TAJO-1727_2.patch
        against master revision release-0.9.0-rc0-445-gfd4a3f8.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-core tajo-core-tests.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/855//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/855//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/855//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12754372/TAJO-1727_2.patch against master revision release-0.9.0-rc0-445-gfd4a3f8. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core tajo-core-tests. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/855//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/855//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/855//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-138421605

        External table must requires a location URL. So, it does not need a tablespace clause.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-138421605 External table must requires a location URL. So, it does not need a tablespace clause.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-138449284

        Thanks @hyunsik .
        I've just updated the patch using your comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-138449284 Thanks @hyunsik . I've just updated the patch using your comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/732#discussion_r39023306

        — Diff: tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java —
        @@ -1285,6 +1285,10 @@ public Expr visitCreate_table_statement(Create_table_statementContext ctx) {
        if (checkIfExist(ctx.EXTERNAL())) {
        createTable.setExternal();

        + if (checkIfExist(ctx.TABLESPACE())) {
        + throw new TajoRuntimeException(new SQLSyntaxError("CREATE EXTERNAL TABLE cannot be used with tablespace"));
        — End diff –

        It would be better if Tablespace clause is a subject. So, I'd like to recommend ``Tablespace clause is not allowed for an external table``.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/732#discussion_r39023306 — Diff: tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java — @@ -1285,6 +1285,10 @@ public Expr visitCreate_table_statement(Create_table_statementContext ctx) { if (checkIfExist(ctx.EXTERNAL())) { createTable.setExternal(); + if (checkIfExist(ctx.TABLESPACE())) { + throw new TajoRuntimeException(new SQLSyntaxError("CREATE EXTERNAL TABLE cannot be used with tablespace")); — End diff – It would be better if Tablespace clause is a subject. So, I'd like to recommend ``Tablespace clause is not allowed for an external table``.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/732#discussion_r39023547

        — Diff: tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java —
        @@ -1293,6 +1297,9 @@ public Expr visitCreate_table_statement(Create_table_statementContext ctx) {
        if (checkIfExist(ctx.LOCATION()))

        { String uri = stripQuote(ctx.uri.getText()); createTable.setLocation(uri); + }

        else {
        + throw new TajoRuntimeException(new SQLSyntaxError("EXTERNAL and LOCATION clauses need to be specified " +
        — End diff –

        How about ``LOCATION clause must be required for an external table.``?

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/732#discussion_r39023547 — Diff: tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java — @@ -1293,6 +1297,9 @@ public Expr visitCreate_table_statement(Create_table_statementContext ctx) { if (checkIfExist(ctx.LOCATION())) { String uri = stripQuote(ctx.uri.getText()); createTable.setLocation(uri); + } else { + throw new TajoRuntimeException(new SQLSyntaxError("EXTERNAL and LOCATION clauses need to be specified " + — End diff – How about ``LOCATION clause must be required for an external table.``?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-138855963

        +1

        The patch looks good to me. It's right validation. If you agree with my comments, you can commit it after you reflect my comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-138855963 +1 The patch looks good to me. It's right validation. If you agree with my comments, you can commit it after you reflect my comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-138917645

        Please hold on a second.
        This patch lacks any test. Would you add it?
        In addition, the issue title is mismatched with the patch.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-138917645 Please hold on a second. This patch lacks any test. Would you add it? In addition, the issue title is mismatched with the patch.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-138921890

        I agree with Jihoon's comment. The title may be matter. For unit tests, I'm going to add some unit tests after I improve the exception system with the better error message. So, it can be skip if you guys agree with my plan.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-138921890 I agree with Jihoon's comment. The title may be matter. For unit tests, I'm going to add some unit tests after I improve the exception system with the better error message. So, it can be skip if you guys agree with my plan.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-139088521

        Thanks guys. I also think that the tile is mismatched.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-139088521 Thanks guys. I also think that the tile is mismatched.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-139104768

        I updated the title and added unit test cases.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-139104768 I updated the title and added unit test cases.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-139236887

        Thanks @jihoonson

        I've reflected your comments and rebased.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-139236887 Thanks @jihoonson I've reflected your comments and rebased.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-139266237

        +1 looks good to me!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-139266237 +1 looks good to me!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/732#issuecomment-139382323

        Thanks @jihoonson

        I'll ship it.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/732#issuecomment-139382323 Thanks @jihoonson I'll ship it.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #499 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/499/)
        TAJO-1727: Avoid to create external table using TableSpace. (blrunner: rev 238f0b3662b919e9b20bdc0bcda647c0d0650ffc)

        • CHANGES
        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql
        • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #499 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/499/ ) TAJO-1727 : Avoid to create external table using TableSpace. (blrunner: rev 238f0b3662b919e9b20bdc0bcda647c0d0650ffc) CHANGES tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/732

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/732
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-0.11.0-build #34 (See https://builds.apache.org/job/Tajo-0.11.0-build/34/)
        TAJO-1727: Avoid to create external table using TableSpace. (blrunner: rev 785352f09701e392b4010d52aa675a5f6d8155d8)

        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java
        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.11.0-build #34 (See https://builds.apache.org/job/Tajo-0.11.0-build/34/ ) TAJO-1727 : Avoid to create external table using TableSpace. (blrunner: rev 785352f09701e392b4010d52aa675a5f6d8155d8) tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql CHANGES tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-build #857 (See https://builds.apache.org/job/Tajo-master-build/857/)
        TAJO-1727: Avoid to create external table using TableSpace. (blrunner: rev 238f0b3662b919e9b20bdc0bcda647c0d0650ffc)

        • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java
        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql
        • CHANGES
        • tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql
        • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-build #857 (See https://builds.apache.org/job/Tajo-master-build/857/ ) TAJO-1727 : Avoid to create external table using TableSpace. (blrunner: rev 238f0b3662b919e9b20bdc0bcda647c0d0650ffc) tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestQueryValidation.java tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_with_tablespace.sql CHANGES tajo-core-tests/src/test/resources/queries/TestQueryValidation/create_external_table_without_location.sql tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java

          People

          • Assignee:
            blrunner Jaehwa Jung
            Reporter:
            blrunner Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development