Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1109

Separate SQL Statements from Catalog Stores

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      When developing the additional catalog stores for another database systems, it is needed to add additional sql statements such as triggers and indexes. These sql statements could increase the code size of catalog store, and when database schema has changed, it may affect the catalog store source codes. I feel that it is needed to separate the sql statements from the java source codes.

      1. DBMSSchemaDefinition.xsd
        6 kB
        Jihun Kang
      2. derby.xml
        6 kB
        Jihun Kang

        Activity

        Hide
        ykrips Jihun Kang added a comment -

        These files are sample schema files for Tajo Catalog stores. Changes on database schema or current database schema sql statements can be placed in a file or multiple files.

        Show
        ykrips Jihun Kang added a comment - These files are sample schema files for Tajo Catalog stores. Changes on database schema or current database schema sql statements can be placed in a file or multiple files.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user ykrips opened a pull request:

        https://github.com/apache/tajo/pull/201

        TAJO-1109: Separate SQL Statements from Catalog Stores

        It has been tested on Embedded derby databases, Oracle 11g, and Oracle 12c.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/ykrips/tajo TAJO-1109

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/201.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #201


        commit 49e021d9fba48dea73f333b865500e6bfac00e08
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-08T15:25:53Z

        TAJO-1093: DateTimeFormat.to_char() is slower than SimpleDateFormat.format()

        commit b49688f424d1b0520f01cd3704163108cf294596
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-12T05:12:48Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c49a84af45fb6672feee1aa7be58431dcb64babc
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:06Z

        Add hamcrest library for fine test on Catalog Server

        commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:12Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c29099024bdce49cc5c65a172d1df3b12753882f
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T10:14:09Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3e7609287777780ba88d4726e3ef3d8548c030ab
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:18:00Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:31:27Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:40:41Z

        TAJO-1109: Separate SQL Statements from Catalog Stores


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user ykrips opened a pull request: https://github.com/apache/tajo/pull/201 TAJO-1109 : Separate SQL Statements from Catalog Stores It has been tested on Embedded derby databases, Oracle 11g, and Oracle 12c. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ykrips/tajo TAJO-1109 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #201 commit 49e021d9fba48dea73f333b865500e6bfac00e08 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-08T15:25:53Z TAJO-1093 : DateTimeFormat.to_char() is slower than SimpleDateFormat.format() commit b49688f424d1b0520f01cd3704163108cf294596 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-12T05:12:48Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c49a84af45fb6672feee1aa7be58431dcb64babc Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:06Z Add hamcrest library for fine test on Catalog Server commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:12Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c29099024bdce49cc5c65a172d1df3b12753882f Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T10:14:09Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3e7609287777780ba88d4726e3ef3d8548c030ab Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:18:00Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:31:27Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:40:41Z TAJO-1109 : Separate SQL Statements from Catalog Stores
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/201#issuecomment-60454535

        Hi @ykrips,

        Even though I'm not familiar with this approach, it definitely seems to make code cleaner as you mentioned in Jira. But, since I'm not familiar with it, I need more time to review it in more detail. It may take few days.

        Also, this patch seems to be stale gone. Could you rebase the patch?

        Thanks in advance

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/201#issuecomment-60454535 Hi @ykrips, Even though I'm not familiar with this approach, it definitely seems to make code cleaner as you mentioned in Jira. But, since I'm not familiar with it, I need more time to review it in more detail. It may take few days. Also, this patch seems to be stale gone. Could you rebase the patch? Thanks in advance
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12673608/derby.xml
        against master revision release-0.9.0-rc0-7-gb636e8b.

        -1 patch. The patch command could not apply the patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/517//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12673608/derby.xml against master revision release-0.9.0-rc0-7-gb636e8b. -1 patch. The patch command could not apply the patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/517//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/201#issuecomment-60554663

        Sure. No problems. I will update this patch with master branch.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/201#issuecomment-60554663 Sure. No problems. I will update this patch with master branch.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/201#issuecomment-60569702

        I'm sorry. I think that I need to figure out why my git repository works not correctly. I also close this pull request and re-open again.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/201#issuecomment-60569702 I'm sorry. I think that I need to figure out why my git repository works not correctly. I also close this pull request and re-open again.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips closed the pull request at:

        https://github.com/apache/tajo/pull/201

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips closed the pull request at: https://github.com/apache/tajo/pull/201
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user ykrips opened a pull request:

        https://github.com/apache/tajo/pull/216

        TAJO-1109: Separate SQL Statements from Catalog Stores

        This is a re-created pull request for this request, https://github.com/apache/tajo/pull/201.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/ykrips/tajo TAJO-1109

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/216.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #216


        commit 49e021d9fba48dea73f333b865500e6bfac00e08
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-08T15:25:53Z

        TAJO-1093: DateTimeFormat.to_char() is slower than SimpleDateFormat.format()

        commit b49688f424d1b0520f01cd3704163108cf294596
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-12T05:12:48Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c49a84af45fb6672feee1aa7be58431dcb64babc
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:06Z

        Add hamcrest library for fine test on Catalog Server

        commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:12Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c29099024bdce49cc5c65a172d1df3b12753882f
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T10:14:09Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3e7609287777780ba88d4726e3ef3d8548c030ab
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:18:00Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:31:27Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:40:41Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit d6d0055abc5a0a6a7e44aaaae3914301ea4750d9
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T08:11:18Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit a4a57385eab62d74382b83324d0d550e34e130c1
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-17T00:18:29Z

        It is changed for strict type checking on PostgreSQL

        commit f0ac6061923f0c3900b4b6ac178e348747f6dff3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-20T01:03:29Z

        changed binary string format on Oracle Store

        commit f04790e52d5ba3370443e0491d83a35d50763153
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:06Z

        Add hamcrest library for fine test on Catalog Server

        commit 1468f52e80bb8d599f494f77a82f35e2310fe9a0
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T10:14:09Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 40d6eb8b83dfe871f52aa474e61e2b7f73d98764
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:18:00Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 80083829049f46be8c1cf65eab95c7385b003d65
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:31:27Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 2950cb4c10a58210364717a3d6dcbcb03aa275df
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:40:41Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 64aa44051b5cfbc3883f937674334e7739ec4dc8
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T08:11:18Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 9e35aeb39ed505e734668439d5475320005b3fc8
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-17T00:18:29Z

        It is changed for strict type checking on PostgreSQL

        commit d76a4e4732fee0ea738ecae828551929035a1836
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-20T01:03:29Z

        changed binary string format on Oracle Store

        commit 37c85d9b2c00067cb6b7cf70a8f46629b350d9b5
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-27T09:51:05Z

        rebasing current patch


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user ykrips opened a pull request: https://github.com/apache/tajo/pull/216 TAJO-1109 : Separate SQL Statements from Catalog Stores This is a re-created pull request for this request, https://github.com/apache/tajo/pull/201 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/ykrips/tajo TAJO-1109 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/216.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #216 commit 49e021d9fba48dea73f333b865500e6bfac00e08 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-08T15:25:53Z TAJO-1093 : DateTimeFormat.to_char() is slower than SimpleDateFormat.format() commit b49688f424d1b0520f01cd3704163108cf294596 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-12T05:12:48Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c49a84af45fb6672feee1aa7be58431dcb64babc Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:06Z Add hamcrest library for fine test on Catalog Server commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:12Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c29099024bdce49cc5c65a172d1df3b12753882f Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T10:14:09Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3e7609287777780ba88d4726e3ef3d8548c030ab Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:18:00Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:31:27Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:40:41Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit d6d0055abc5a0a6a7e44aaaae3914301ea4750d9 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T08:11:18Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit a4a57385eab62d74382b83324d0d550e34e130c1 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-17T00:18:29Z It is changed for strict type checking on PostgreSQL commit f0ac6061923f0c3900b4b6ac178e348747f6dff3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-20T01:03:29Z changed binary string format on Oracle Store commit f04790e52d5ba3370443e0491d83a35d50763153 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:06Z Add hamcrest library for fine test on Catalog Server commit 1468f52e80bb8d599f494f77a82f35e2310fe9a0 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T10:14:09Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 40d6eb8b83dfe871f52aa474e61e2b7f73d98764 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:18:00Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 80083829049f46be8c1cf65eab95c7385b003d65 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:31:27Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 2950cb4c10a58210364717a3d6dcbcb03aa275df Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:40:41Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 64aa44051b5cfbc3883f937674334e7739ec4dc8 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T08:11:18Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 9e35aeb39ed505e734668439d5475320005b3fc8 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-17T00:18:29Z It is changed for strict type checking on PostgreSQL commit d76a4e4732fee0ea738ecae828551929035a1836 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-20T01:03:29Z changed binary string format on Oracle Store commit 37c85d9b2c00067cb6b7cf70a8f46629b350d9b5 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-27T09:51:05Z rebasing current patch
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-61223240

        Could you give more background or materials to understand this technique? Actually, I'm not familiar with this technique

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-61223240 Could you give more background or materials to understand this technique? Actually, I'm not familiar with this technique
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-61223278

        But, I definitely know this is very cool and this will make code simpler than before.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-61223278 But, I definitely know this is very cool and this will make code simpler than before.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-61229439

        Well, I did not remember who stated this technique. I feel that it may comes for a need for separation of roles for DBA and programmer, or it may comes for easier maintenance. At first, I would liked to add codes for sql separations on catalog stores, but I added some codes for altering and deleting databse objects because I prefer to automation on database schema modification. Also, I found a similar document in a below link.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-61229439 Well, I did not remember who stated this technique. I feel that it may comes for a need for separation of roles for DBA and programmer, or it may comes for easier maintenance. At first, I would liked to add codes for sql separations on catalog stores, but I added some codes for altering and deleting databse objects because I prefer to automation on database schema modification. Also, I found a similar document in a below link. http://www.javapractices.com/topic/TopicAction.do?Id=105
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-61599711

        Thank you for sharing the materials.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-61599711 Thank you for sharing the materials.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r19790604

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        Upgrade support is a nice approach. BTW, we have used ```tajo-dump``` for upgrading catalog stores. You can see the documentation of this feature at http://tajo.apache.org/docs/current/backup_and_restore/catalog.html.

        The main reason why Tajo uses ```tajo-dump``` approach is to mitigate the maintenance effort of schema changes between versions. With your approach, we need to maintain schema changes for each RDBMS catalog store. ```tajo_dump``` allows us to reuse the part of DDL statement execution in Tajo, and we just need to keep DDL DUMP code.

        Even through currently schemas for different RDBMSs are not much different to one another, the schemas will much differ as we optimize and specialize CatalogStore. Thus, the maintenance effort of schema changes between different versions would be costly.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r19790604 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – Upgrade support is a nice approach. BTW, we have used ```tajo-dump``` for upgrading catalog stores. You can see the documentation of this feature at http://tajo.apache.org/docs/current/backup_and_restore/catalog.html . The main reason why Tajo uses ```tajo-dump``` approach is to mitigate the maintenance effort of schema changes between versions. With your approach, we need to maintain schema changes for each RDBMS catalog store. ```tajo_dump``` allows us to reuse the part of DDL statement execution in Tajo, and we just need to keep DDL DUMP code. Even through currently schemas for different RDBMSs are not much different to one another, the schemas will much differ as we optimize and specialize CatalogStore. Thus, the maintenance effort of schema changes between different versions would be costly.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-61609265

        I verified 'mvn clean install' and I tested the patch on PostgreSQL server. It works well. The patch looks nice to me. I just leave some trivial comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-61609265 I verified 'mvn clean install' and I tested the patch on PostgreSQL server. It works well. The patch looks nice to me. I just leave some trivial comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r19864361

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        I understand your concerns about maintaining upgrade sql fragementations in tajo project. Yes. It is difficult to keep the current version of sql codes. However, code about upgrade support is derived from my user experience. When I was a newbie in tajo, it is difficult for me to finding tajo tables and any related schema objects in existing RDBMS. So, can I say alternatives on this? How about creating schema objects in RDBMS? It is very useful for database administrators or newbies.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r19864361 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – I understand your concerns about maintaining upgrade sql fragementations in tajo project. Yes. It is difficult to keep the current version of sql codes. However, code about upgrade support is derived from my user experience. When I was a newbie in tajo, it is difficult for me to finding tajo tables and any related schema objects in existing RDBMS. So, can I say alternatives on this? How about creating schema objects in RDBMS? It is very useful for database administrators or newbies.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r20110576

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        Hi @ykrips,

        That's nice idea. The documentation about schema objects would be very helpful to developers and operation engineers.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r20110576 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – Hi @ykrips, That's nice idea. The documentation about schema objects would be very helpful to developers and operation engineers.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r20112082

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        I believe that ```tajo_dump``` approach is productive to us in terms of development and more safe for upgrading.

        With ```tajo_dump```, the instruction of Tajo migration is as follows:

        1. execute ```tajo_dump``` to dump your all catalog information to *Tajo DDL statements* as a plan text file (e.g., backup.sql).
        2. create new database in your catalog store (PostgreSQL or MySQL) for migration
        3. change the catalog store config in ```catalog-site.xml``` in order to use newly created database
        4. restart a Tajo cluster
        5. run ```tsql -f backup.sql``` to execute a sequence of *Tajo DDL statements* from the backup.sql

        This approach naturally allows users to keep original catalog tables without additional database backup. Even if upgrade is failed, the original catalogs would be safe.

        Another motivation of ```tajo-dump``` approach allows us to change catalog schemas without considering schema upgrades. Actually, our catalog is still evolving rapidly. It may be important to us in this time.

        This is my opinion. I'm open to any discussion.

        Thank you for your work.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r20112082 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – I believe that ```tajo_dump``` approach is productive to us in terms of development and more safe for upgrading. With ```tajo_dump```, the instruction of Tajo migration is as follows: 1. execute ```tajo_dump``` to dump your all catalog information to * Tajo DDL statements * as a plan text file (e.g., backup.sql). 2. create new database in your catalog store (PostgreSQL or MySQL) for migration 3. change the catalog store config in ```catalog-site.xml``` in order to use newly created database 4. restart a Tajo cluster 5. run ```tsql -f backup.sql``` to execute a sequence of * Tajo DDL statements * from the backup.sql This approach naturally allows users to keep original catalog tables without additional database backup. Even if upgrade is failed, the original catalogs would be safe. Another motivation of ```tajo-dump``` approach allows us to change catalog schemas without considering schema upgrades. Actually, our catalog is still evolving rapidly. It may be important to us in this time. This is my opinion. I'm open to any discussion. Thank you for your work.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r20127482

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        I think that difference on upgrade approach might come with who will perform upgrade processes on Tajo. My approach considers software engineers who do not know about RDBMS or do not have the privileges on RDBMS. However, with your approach, it is needed to engage database administrator when upgrading catalog information. Both approaches have pros and cons, but if catalog schema has changed rapidly, your approach may be suitable. My approach will be difficult to maintain the catalog schema if these changes occur frequently.
        I will restore source codes on version check, and will commit soon.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r20127482 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – I think that difference on upgrade approach might come with who will perform upgrade processes on Tajo. My approach considers software engineers who do not know about RDBMS or do not have the privileges on RDBMS. However, with your approach, it is needed to engage database administrator when upgrading catalog information. Both approaches have pros and cons, but if catalog schema has changed rapidly, your approach may be suitable. My approach will be difficult to maintain the catalog schema if these changes occur frequently. I will restore source codes on version check, and will commit soon.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/216#issuecomment-62496616

        When I rebasing this commit, it also add other commits which are not related to this issue. I'm sorry for this interruption. I will close and re-open again.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/216#issuecomment-62496616 When I rebasing this commit, it also add other commits which are not related to this issue. I'm sorry for this interruption. I will close and re-open again.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips closed the pull request at:

        https://github.com/apache/tajo/pull/216

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips closed the pull request at: https://github.com/apache/tajo/pull/216
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user ykrips opened a pull request:

        https://github.com/apache/tajo/pull/236

        TAJO-1109: Separate SQL Statements from Catalog Stores

        Re-created a pull request for TAJO-1109.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/ykrips/tajo TAJO-1109

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/236.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #236


        commit 49e021d9fba48dea73f333b865500e6bfac00e08
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-08T15:25:53Z

        TAJO-1093: DateTimeFormat.to_char() is slower than SimpleDateFormat.format()

        commit b49688f424d1b0520f01cd3704163108cf294596
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-12T05:12:48Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c49a84af45fb6672feee1aa7be58431dcb64babc
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:06Z

        Add hamcrest library for fine test on Catalog Server

        commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:12Z

        Merge remote-tracking branch 'upstream/master' into TAJO-1109

        commit c29099024bdce49cc5c65a172d1df3b12753882f
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T10:14:09Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3e7609287777780ba88d4726e3ef3d8548c030ab
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:18:00Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:31:27Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:40:41Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit d6d0055abc5a0a6a7e44aaaae3914301ea4750d9
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T08:11:18Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit a4a57385eab62d74382b83324d0d550e34e130c1
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-17T00:18:29Z

        It is changed for strict type checking on PostgreSQL

        commit f0ac6061923f0c3900b4b6ac178e348747f6dff3
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-20T01:03:29Z

        changed binary string format on Oracle Store

        commit f04790e52d5ba3370443e0491d83a35d50763153
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T06:31:06Z

        Add hamcrest library for fine test on Catalog Server

        commit 1468f52e80bb8d599f494f77a82f35e2310fe9a0
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-14T10:14:09Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 40d6eb8b83dfe871f52aa474e61e2b7f73d98764
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:18:00Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 80083829049f46be8c1cf65eab95c7385b003d65
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:31:27Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 2950cb4c10a58210364717a3d6dcbcb03aa275df
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T06:40:41Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 64aa44051b5cfbc3883f937674334e7739ec4dc8
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-15T08:11:18Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit 9e35aeb39ed505e734668439d5475320005b3fc8
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-17T00:18:29Z

        It is changed for strict type checking on PostgreSQL

        commit d76a4e4732fee0ea738ecae828551929035a1836
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-20T01:03:29Z

        changed binary string format on Oracle Store

        commit 37c85d9b2c00067cb6b7cf70a8f46629b350d9b5
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-10-27T09:51:05Z

        rebasing current patch

        commit fd4d6970e7a79bd895e49b0150475ce8e6c83334
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-11-11T03:18:48Z

        Merge branch 'TAJO-1109' of https://github.com/ykrips/tajo into TAJO-1109

        commit 1eb833ce41524c80ca17e88a5f908bd1b922565f
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-11-11T03:21:12Z

        TAJO-1109: Separate SQL Statements from Catalog Stores

        commit b74da6cede387e0a09acca9a16b6df8594d99dac
        Author: Jihun Kang <ykrips@gmail.com>
        Date: 2014-11-11T04:16:57Z

        TAJO-1109: Separate SQL Statements from Catalog Stores


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user ykrips opened a pull request: https://github.com/apache/tajo/pull/236 TAJO-1109 : Separate SQL Statements from Catalog Stores Re-created a pull request for TAJO-1109 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/ykrips/tajo TAJO-1109 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/236.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #236 commit 49e021d9fba48dea73f333b865500e6bfac00e08 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-08T15:25:53Z TAJO-1093 : DateTimeFormat.to_char() is slower than SimpleDateFormat.format() commit b49688f424d1b0520f01cd3704163108cf294596 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-12T05:12:48Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c49a84af45fb6672feee1aa7be58431dcb64babc Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:06Z Add hamcrest library for fine test on Catalog Server commit 9496fcde7a2810c0e0575b0c5d483adac28f8bf3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:12Z Merge remote-tracking branch 'upstream/master' into TAJO-1109 commit c29099024bdce49cc5c65a172d1df3b12753882f Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T10:14:09Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3e7609287777780ba88d4726e3ef3d8548c030ab Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:18:00Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 3559108bd2efcd8d32730dd3d2bea7cd845490c3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:31:27Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 2c895ea3253d58ffd43d3f4a70dbb093099c4888 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:40:41Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit d6d0055abc5a0a6a7e44aaaae3914301ea4750d9 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T08:11:18Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit a4a57385eab62d74382b83324d0d550e34e130c1 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-17T00:18:29Z It is changed for strict type checking on PostgreSQL commit f0ac6061923f0c3900b4b6ac178e348747f6dff3 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-20T01:03:29Z changed binary string format on Oracle Store commit f04790e52d5ba3370443e0491d83a35d50763153 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T06:31:06Z Add hamcrest library for fine test on Catalog Server commit 1468f52e80bb8d599f494f77a82f35e2310fe9a0 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-14T10:14:09Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 40d6eb8b83dfe871f52aa474e61e2b7f73d98764 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:18:00Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 80083829049f46be8c1cf65eab95c7385b003d65 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:31:27Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 2950cb4c10a58210364717a3d6dcbcb03aa275df Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T06:40:41Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 64aa44051b5cfbc3883f937674334e7739ec4dc8 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-15T08:11:18Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit 9e35aeb39ed505e734668439d5475320005b3fc8 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-17T00:18:29Z It is changed for strict type checking on PostgreSQL commit d76a4e4732fee0ea738ecae828551929035a1836 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-20T01:03:29Z changed binary string format on Oracle Store commit 37c85d9b2c00067cb6b7cf70a8f46629b350d9b5 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-10-27T09:51:05Z rebasing current patch commit fd4d6970e7a79bd895e49b0150475ce8e6c83334 Author: Jihun Kang <ykrips@gmail.com> Date: 2014-11-11T03:18:48Z Merge branch ' TAJO-1109 ' of https://github.com/ykrips/tajo into TAJO-1109 commit 1eb833ce41524c80ca17e88a5f908bd1b922565f Author: Jihun Kang <ykrips@gmail.com> Date: 2014-11-11T03:21:12Z TAJO-1109 : Separate SQL Statements from Catalog Stores commit b74da6cede387e0a09acca9a16b6df8594d99dac Author: Jihun Kang <ykrips@gmail.com> Date: 2014-11-11T04:16:57Z TAJO-1109 : Separate SQL Statements from Catalog Stores
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/216#discussion_r20141159

        — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java —
        @@ -176,42 +202,47 @@ public Connection getConnection()

        { return conn; }
        • private void verifySchemaVersion() throws CatalogException {
          + private int getSchemaVersion() {
          Connection conn = null;
          PreparedStatement pstmt = null;
          ResultSet result = null;
          + int schemaVersion = -1;
          +
          + String sql = "SELECT version FROM META";
          + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + }

        try {

        • String sql = "SELECT version FROM META";
        • if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - }

          -
          conn = getConnection();
          pstmt = conn.prepareStatement(sql);
          result = pstmt.executeQuery();

        • boolean noVersion = !result.next();
          -
        • int schemaVersion = result.getInt(1);
        • if (noVersion || schemaVersion != getDriverVersion()) {
        • LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
        • schemaVersion, getDriverVersion()));
        • LOG.error("=========================================================================");
        • LOG.error("| Catalog Store Migration Is Needed |");
        • LOG.error("=========================================================================");
        • LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |");
        • LOG.error("| Tajo without migration process is only available in some versions. |");
        • LOG.error("| In order to learn how to migration Apache Tajo instance, |");
        • LOG.error("| please refer http://s.apache.org/0_8_migration. |");
        • LOG.error("=========================================================================");
        • throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration.");
          + if (result.next()) { + schemaVersion = result.getInt("VERSION"); }

          } catch (SQLException e)

          { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); }

          finally

          { CatalogUtil.closeQuietly(pstmt, result); }

          +
          + return schemaVersion;
          + }
          +
          + private void verifySchemaVersion() throws CatalogException {
          + int schemaVersion = -1;
          +
          + schemaVersion = getSchemaVersion();
          +
          + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) {
          + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other",
          + schemaVersion, getDriverVersion()));
          + LOG.info("It will start upgrade process on catalog stores.");
          +
          + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion);

            • End diff –

        I agree with your point. Your suggestion would be very helpful to engineers who don't want to manage RDBMSs. When our catalog becomes stable, we can add the upgrade approach.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/216#discussion_r20141159 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java — @@ -176,42 +202,47 @@ public Connection getConnection() { return conn; } private void verifySchemaVersion() throws CatalogException { + private int getSchemaVersion() { Connection conn = null; PreparedStatement pstmt = null; ResultSet result = null; + int schemaVersion = -1; + + String sql = "SELECT version FROM META"; + if (LOG.isDebugEnabled()) { + LOG.debug(sql.toString()); + } try { String sql = "SELECT version FROM META"; if (LOG.isDebugEnabled()) { - LOG.debug(sql.toString()); - } - conn = getConnection(); pstmt = conn.prepareStatement(sql); result = pstmt.executeQuery(); boolean noVersion = !result.next(); - int schemaVersion = result.getInt(1); if (noVersion || schemaVersion != getDriverVersion()) { LOG.error(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", schemaVersion, getDriverVersion())); LOG.error("========================================================================="); LOG.error("| Catalog Store Migration Is Needed |"); LOG.error("========================================================================="); LOG.error("| You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |"); LOG.error("| Tajo without migration process is only available in some versions. |"); LOG.error("| In order to learn how to migration Apache Tajo instance, |"); LOG.error("| please refer http://s.apache.org/0_8_migration . |"); LOG.error("========================================================================="); throw new CatalogException("Migration Needed. Please refer http://s.apache.org/0_8_migration ."); + if (result.next()) { + schemaVersion = result.getInt("VERSION"); } } catch (SQLException e) { - throw new CatalogException(e); + throw new CatalogException(e.getMessage(), e); } finally { CatalogUtil.closeQuietly(pstmt, result); } + + return schemaVersion; + } + + private void verifySchemaVersion() throws CatalogException { + int schemaVersion = -1; + + schemaVersion = getSchemaVersion(); + + if (schemaVersion == -1 || schemaVersion != getDriverVersion()) { + LOG.info(String.format("Catalog version (%d) and current driver version (%d) are mismatch to each other", + schemaVersion, getDriverVersion())); + LOG.info("It will start upgrade process on catalog stores."); + + catalogSchemaManager.upgradeBaseSchema(getConnection(), schemaVersion); End diff – I agree with your point. Your suggestion would be very helpful to engineers who don't want to manage RDBMSs. When our catalog becomes stable, we can add the upgrade approach.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/236#issuecomment-62527806

        Excellent! Here is my +1.

        I've just tested the patch on PostgreSQL and Derby. It works well, and the patch looks nice to me. I'll commit the patch to master branch if there is no objection until tomorrow.

        FYI, this pull request continues to the following pull requests.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/236#issuecomment-62527806 Excellent! Here is my +1. I've just tested the patch on PostgreSQL and Derby. It works well, and the patch looks nice to me. I'll commit the patch to master branch if there is no objection until tomorrow. FYI, this pull request continues to the following pull requests. https://github.com/apache/tajo/pull/216 https://github.com/apache/tajo/pull/201
        Hide
        hyunsik Hyunsik Choi added a comment -

        I just committed the patch to master branch. Thank you Jihun!

        Show
        hyunsik Hyunsik Choi added a comment - I just committed the patch to master branch. Thank you Jihun!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #87 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/87/)
        TAJO-1109: Separate SQL Statements from Catalog Stores. (Jihun Kang via hyunsik) (hyunsik: rev 005556850e60f14d5e11ca74035b195461f31d29)

        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases_idx.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/querytest/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SQLObject.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/StoreObject.java
        • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/store/TestXMLCatalogSchemaManager.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/XMLCatalogSchemaManager.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tables.sql
        • tajo-catalog/tajo-catalog-server/pom.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tablespaces.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObject.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/DBMSSchemaDefinition.xsd
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/upgradetest/base_version_2.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/OracleStore.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/BaseSchema.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/PostgreSQLStore.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObjectType.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_1.xml
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/loadtest/derby.xml
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_2.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SchemaPatch.java
        • CHANGES
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #87 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/87/ ) TAJO-1109 : Separate SQL Statements from Catalog Stores. (Jihun Kang via hyunsik) (hyunsik: rev 005556850e60f14d5e11ca74035b195461f31d29) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases_idx.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/indexes.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partition_methods.sql tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/querytest/derby.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SQLObject.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/StoreObject.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/store/TestXMLCatalogSchemaManager.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/indexes.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/XMLCatalogSchemaManager.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tables.sql tajo-catalog/tajo-catalog-server/pom.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tablespaces.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObject.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/DBMSSchemaDefinition.xsd tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/upgradetest/base_version_2.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/OracleStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/BaseSchema.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/PostgreSQLStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObjectType.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/table_properties.sql tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_1.xml tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/loadtest/derby.xml tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_2.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/stats.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/stats.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SchemaPatch.java CHANGES
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #445 (See https://builds.apache.org/job/Tajo-master-build/445/)
        TAJO-1109: Separate SQL Statements from Catalog Stores. (Jihun Kang via hyunsik) (hyunsik: rev 005556850e60f14d5e11ca74035b195461f31d29)

        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/BaseSchema.java
        • tajo-catalog/tajo-catalog-server/pom.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partitions.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tablespaces.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/PostgreSQLStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partition_methods.sql
        • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/store/TestXMLCatalogSchemaManager.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/columns.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/loadtest/derby.xml
        • CHANGES
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObject.java
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_2.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SQLObject.java
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/upgradetest/base_version_2.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/stats.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/DBMSSchemaDefinition.xsd
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/StoreObject.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/XMLCatalogSchemaManager.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tables.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_1.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObjectType.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/OracleStore.java
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases_idx.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/indexes.sql
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SchemaPatch.java
        • tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/querytest/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #445 (See https://builds.apache.org/job/Tajo-master-build/445/ ) TAJO-1109 : Separate SQL Statements from Catalog Stores. (Jihun Kang via hyunsik) (hyunsik: rev 005556850e60f14d5e11ca74035b195461f31d29) tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/BaseSchema.java tajo-catalog/tajo-catalog-server/pom.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partitions.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/stats.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tablespaces.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/PostgreSQLStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/partition_methods.sql tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/store/TestXMLCatalogSchemaManager.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/columns.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/loadtest/derby.xml CHANGES tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObject.java tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_2.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SQLObject.java tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/upgradetest/base_version_2.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/stats.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/DBMSSchemaDefinition.xsd tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/StoreObject.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/XMLCatalogSchemaManager.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/tables.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/indexes.sql tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/mergetest/base_version_1.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/DatabaseObjectType.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/OracleStore.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/databases_idx.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/indexes.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/object/SchemaPatch.java tajo-catalog/tajo-catalog-server/src/test/resources/schemas/derbytest/querytest/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/236#issuecomment-62655161

        Hi @ykrips,

        Could you close this issue? I included #201 instead of #236 in the commit log. The number plays a role to automatically close the corresponding pull request.

        https://github.com/apache/tajo/commit/005556850e60f14d5e11ca74035b195461f31d29

        Thanks in advance.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/236#issuecomment-62655161 Hi @ykrips, Could you close this issue? I included #201 instead of #236 in the commit log. The number plays a role to automatically close the corresponding pull request. https://github.com/apache/tajo/commit/005556850e60f14d5e11ca74035b195461f31d29 Thanks in advance.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/236#issuecomment-62655923

        Hi @hyunsik,

        No problems. I will close this issue.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/236#issuecomment-62655923 Hi @hyunsik, No problems. I will close this issue.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips closed the pull request at:

        https://github.com/apache/tajo/pull/236

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips closed the pull request at: https://github.com/apache/tajo/pull/236

          People

          • Assignee:
            ykrips Jihun Kang
            Reporter:
            ykrips Jihun Kang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development