Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      Currently, CatalogStore is designed to read schema xml file on initial mode. But MySQLStore and MariaDBStore read SQL statement files. Thus we need to clean up these files.

      1. TAJO-1495.patch
        42 kB
        Jaehwa Jung

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user blrunner opened a pull request:

        https://github.com/apache/tajo/pull/491

        TAJO-1495: Clean up CatalogStore

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/blrunner/tajo TAJO-1495

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/491.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #491


        commit 92b121122cf07c84517e4cfecc6e28bf165df771
        Author: JaeHwa Jung <blrunner@apache.org>
        Date: 2015-04-01T05:35:19Z

        TAJO-1495: Clean up CatalogStore


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user blrunner opened a pull request: https://github.com/apache/tajo/pull/491 TAJO-1495 : Clean up CatalogStore You can merge this pull request into a Git repository by running: $ git pull https://github.com/blrunner/tajo TAJO-1495 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/491.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #491 commit 92b121122cf07c84517e4cfecc6e28bf165df771 Author: JaeHwa Jung <blrunner@apache.org> Date: 2015-04-01T05:35:19Z TAJO-1495 : Clean up CatalogStore
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708609/TAJO-1495.patch
        against master revision release-0.9.0-rc0-227-g487a0e5.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-server.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/689//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/689//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-catalog-server.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/689//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708609/TAJO-1495.patch against master revision release-0.9.0-rc0-227-g487a0e5. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-server. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/689//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/689//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-catalog-server.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/689//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88361542

        For the reference, I finished this successfully on MySQL, MariaDB, PostgreSQL.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88361542 For the reference, I finished this successfully on MySQL, MariaDB, PostgreSQL.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88361761

        @ykrips
        Could you review this PR ?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88361761 @ykrips Could you review this PR ?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88370093

        @jinnosy.
        Sure. no problems. However, I don't have much knowledge on mysql and mariadb, and need lead time to make over for these dbms. If it is not an urgent issue, could you wait for some time to recover my knowledge on mysql and mariadb?

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88370093 @jinnosy. Sure. no problems. However, I don't have much knowledge on mysql and mariadb, and need lead time to make over for these dbms. If it is not an urgent issue, could you wait for some time to recover my knowledge on mysql and mariadb?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88370874

        Yes, I can. This PR is not an urgent issue. please go ahead
        Thank you!!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88370874 Yes, I can. This PR is not an urgent issue. please go ahead Thank you!!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88382391

        Thanks @ykrips and @jinossy

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88382391 Thanks @ykrips and @jinossy
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88799535

        @blrunner ,
        Column size of `VALUE_` on `OPTIONS` table has been increased to 4000 bytes, could you change these size in your patch? Patch for TAJO-1360 already uploaded earlier, so I feel that it could be better to apply first to our master branch.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88799535 @blrunner , Column size of `VALUE_` on `OPTIONS` table has been increased to 4000 bytes, could you change these size in your patch? Patch for TAJO-1360 already uploaded earlier, so I feel that it could be better to apply first to our master branch.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88804247

        @ykrips

        Thanks for your comments. I've just updated it.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88804247 @ykrips Thanks for your comments. I've just updated it.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88833237

        +1
        TestCatalog testcase has passed on my derby, mysql, mariadb, postgresql, and oracle database. Also, table schema verified by using its own command-line commands.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88833237 +1 TestCatalog testcase has passed on my derby, mysql, mariadb, postgresql, and oracle database. Also, table schema verified by using its own command-line commands.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/491#issuecomment-88835900

        @ykrips

        Thanks for your kind and quick review.
        I'll ship it now.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/491#issuecomment-88835900 @ykrips Thanks for your kind and quick review. I'll ship it now.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/491

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/491
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #646 (See https://builds.apache.org/job/Tajo-master-build/646/)
        TAJO-1495: Clean up CatalogStore. (jaehwa) (blrunner: rev bda2d6236df1db352206e831b0da93b31de701bf)

        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tablespaces.sql
        • CHANGES
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/common/meta.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tablespaces.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MySQLStore.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractMySQLMariaDBStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tables.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tables.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_keys.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_keys.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MariaDBStore.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #646 (See https://builds.apache.org/job/Tajo-master-build/646/ ) TAJO-1495 : Clean up CatalogStore. (jaehwa) (blrunner: rev bda2d6236df1db352206e831b0da93b31de701bf) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tablespaces.sql CHANGES tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/databases.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/common/meta.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/stats.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tablespaces.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/stats.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MySQLStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractMySQLMariaDBStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tables.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tables.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/indexes.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_methods.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_keys.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_keys.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/databases.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/indexes.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_methods.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MariaDBStore.java
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-CODEGEN-build #283 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/283/)
        TAJO-1495: Clean up CatalogStore. (jaehwa) (blrunner: rev bda2d6236df1db352206e831b0da93b31de701bf)

        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tables.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tablespaces.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql
        • CHANGES
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_keys.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/common/meta.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MariaDBStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tablespaces.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tables.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MySQLStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_keys.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractMySQLMariaDBStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/indexes.sql
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #283 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/283/ ) TAJO-1495 : Clean up CatalogStore. (jaehwa) (blrunner: rev bda2d6236df1db352206e831b0da93b31de701bf) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tables.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/tablespaces.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql CHANGES tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_keys.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_methods.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/common/meta.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MariaDBStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/indexes.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/databases.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tablespaces.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/tables.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/stats.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/MySQLStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/stats.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/databases.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partition_keys.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractMySQLMariaDBStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/partition_methods.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/indexes.sql

          People

          • Assignee:
            blrunner Jaehwa Jung
            Reporter:
            blrunner Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development