Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1360

VALUES_ field in OPTIONS table of catalog store should be longer.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0, 0.10.1
    • Component/s: Catalog
    • Labels:
      None

      Description

      Table properties for each table are maintained in OPTIONS table in catalog store. Each key and value for each table property use KEY_ and VALUE_ fields as follows:

      CREATE TABLE OPTIONS (
        TID INT NOT NULL,
        KEY_ VARCHAR(255) BINARY NOT NULL,
        VALUE_ VARCHAR(255) NOT NULL,
        PRIMARY KEY (TID, KEY_),
        FOREIGN KEY (TID) REFERENCES TABLES (TID) ON DELETE CASCADE
      )
      

      Newly adopted Hbase storage uses the table property columns to maintain the column map between a HBase table and Tajo table. VALUE_ is likely to cause 'Data too long' exception because its maximum length is just 256.

      So, we need to increase the maximum size. I think that 8000 would be proper because some RDBMSs supports VARCHAR types up to 8000 chars as far as I know.

        Activity

        Hide
        blrunner Jaehwa Jung added a comment -

        I've just committed this to the branch-0.10.1, too.

        Show
        blrunner Jaehwa Jung added a comment - I've just committed this to the branch-0.10.1, too.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #645 (See https://builds.apache.org/job/Tajo-master-build/645/)
        TAJO-1360: VALUES_ field in OPTIONS table of catalog store should be longer. (jihun: rev d6735fb929daf86df3c8adb729a4e64bce95462b)

        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • CHANGES
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #645 (See https://builds.apache.org/job/Tajo-master-build/645/ ) TAJO-1360 : VALUES_ field in OPTIONS table of catalog store should be longer. (jihun: rev d6735fb929daf86df3c8adb729a4e64bce95462b) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml CHANGES tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-CODEGEN-build #282 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/282/)
        TAJO-1360: VALUES_ field in OPTIONS table of catalog store should be longer. (jihun: rev d6735fb929daf86df3c8adb729a4e64bce95462b)

        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
        • CHANGES
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
        • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #282 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/282/ ) TAJO-1360 : VALUES_ field in OPTIONS table of catalog store should be longer. (jihun: rev d6735fb929daf86df3c8adb729a4e64bce95462b) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/table_properties.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml CHANGES tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/table_properties.sql
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/436

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/436
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/436#issuecomment-88780687

        Hello All,
        +1
        This patch was tested with Derby, MySQL 5.1.17, MariaDB 10.0.17, PostgreSQL 8.4.18, and Oracle 12.1.0.2.0, and worked well as expected. I will commit it in a short time.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/436#issuecomment-88780687 Hello All, +1 This patch was tested with Derby, MySQL 5.1.17, MariaDB 10.0.17, PostgreSQL 8.4.18, and Oracle 12.1.0.2.0, and worked well as expected. I will commit it in a short time.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/436#issuecomment-88677501

        @ykrips I would greatly appreciate it if you finish up this request.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/436#issuecomment-88677501 @ykrips I would greatly appreciate it if you finish up this request.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/436#issuecomment-88412423

        @hyunsik ,
        If you are unable to work on this pull request, can I finish up this request?
        This issue is related to another pull request, #491 .

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/436#issuecomment-88412423 @hyunsik , If you are unable to work on this pull request, can I finish up this request? This issue is related to another pull request, #491 .
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-83269616

        @hyunsik @dongjoon-hyun @ykrips
        Please see https://github.com/apache/tajo/pull/436
        I'm sorry to make you confused

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-83269616 @hyunsik @dongjoon-hyun @ykrips Please see https://github.com/apache/tajo/pull/436 I'm sorry to make you confused
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/436#issuecomment-83269243

        @hyunsik @dongjoon-hyun @ykrips
        I'm sorry to make you confused. Thank you.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/436#issuecomment-83269243 @hyunsik @dongjoon-hyun @ykrips I'm sorry to make you confused. Thank you.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user charsyam opened a pull request:

        https://github.com/apache/tajo/pull/436

        TAJO-1360: VALUES_ field in OPTIONS table of catalog store should be longer.

        I suggest OPTIONS size as 4000, because max size of oracle varchar2 is just 4000
        I researched docs for these patch.

        maria: https://mariadb.com/kb/en/mariadb/varchar/
        mysql: http://dev.mysql.com/doc/refman/5.7/en/char.html
        postgres: http://www.postgresql.org/docs/8.3/static/datatype-character.html
        derby: https://db.apache.org/derby/docs/10.7/ref/rrefsqlj41207.html
        oracle: https://docs.oracle.com/cd/B28359_01/server.111/b28320/limits001.htm#i287903
        (here varchar2 limit is just 4000)

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/charsyam/tajo feature/TAJO-1360

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/436.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #436


        commit cb4c585388b0efe99394aa8115045b7f8cba0f5b
        Author: clark.kang <clark.kang@kakao.com>
        Date: 2015-03-19T02:21:46Z

        change VALUE_ size


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user charsyam opened a pull request: https://github.com/apache/tajo/pull/436 TAJO-1360 : VALUES_ field in OPTIONS table of catalog store should be longer. I suggest OPTIONS size as 4000, because max size of oracle varchar2 is just 4000 I researched docs for these patch. maria: https://mariadb.com/kb/en/mariadb/varchar/ mysql: http://dev.mysql.com/doc/refman/5.7/en/char.html postgres: http://www.postgresql.org/docs/8.3/static/datatype-character.html derby: https://db.apache.org/derby/docs/10.7/ref/rrefsqlj41207.html oracle: https://docs.oracle.com/cd/B28359_01/server.111/b28320/limits001.htm#i287903 (here varchar2 limit is just 4000) You can merge this pull request into a Git repository by running: $ git pull https://github.com/charsyam/tajo feature/ TAJO-1360 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/436.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #436 commit cb4c585388b0efe99394aa8115045b7f8cba0f5b Author: clark.kang <clark.kang@kakao.com> Date: 2015-03-19T02:21:46Z change VALUE_ size
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam closed the pull request at:

        https://github.com/apache/tajo/pull/413

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam closed the pull request at: https://github.com/apache/tajo/pull/413
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-83267775

        There is something wrong. so I will send it again.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-83267775 There is something wrong. so I will send it again.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-83266479

        @hyunsik I will patch it soon ASAP.
        actually, I forgot it, Thank you for reminding.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-83266479 @hyunsik I will patch it soon ASAP. actually, I forgot it, Thank you for reminding.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-82811255

        Hi @charsyam,

        Any progress? If you are busy now, I can finish this issue.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-82811255 Hi @charsyam, Any progress? If you are busy now, I can finish this issue.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-78406665

        Thank you @ykrips for your comment. I missed them.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-78406665 Thank you @ykrips for your comment. I missed them.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user ykrips commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-78404355

        Since I did move sql statements to xml files for Oracle and PostgreSQL DBMS, but these deprecated sql files for Oracle and PostgreSQL are still remained. Sorry for that. Can you apply these changes to /tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml and /tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml? I will create another issue to delete these deprecated sql files. Without this issue, I don't have any concerns on this patch.

        Show
        githubbot ASF GitHub Bot added a comment - Github user ykrips commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-78404355 Since I did move sql statements to xml files for Oracle and PostgreSQL DBMS, but these deprecated sql files for Oracle and PostgreSQL are still remained. Sorry for that. Can you apply these changes to /tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml and /tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml? I will create another issue to delete these deprecated sql files. Without this issue, I don't have any concerns on this patch.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user dongjoon-hyun commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-78401507

        +1

        Show
        githubbot ASF GitHub Bot added a comment - Github user dongjoon-hyun commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-78401507 +1
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/413#issuecomment-78400305

        +1

        Even though I suggested 8000 chars, 4000 chars also look enough. When we face the same problem due to limited length, we need to investigate new solution.

        If there are no objection until tomorrow, I'll commit it.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/413#issuecomment-78400305 +1 Even though I suggested 8000 chars, 4000 chars also look enough. When we face the same problem due to limited length, we need to investigate new solution. If there are no objection until tomorrow, I'll commit it.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user charsyam opened a pull request:

        https://github.com/apache/tajo/pull/413

        TAJO-1360: VALUES_ field in OPTIONS table of catalog store should be longer.

        I suggest OPTIONS size as 4000, because max size of oracle varchar2 is just 4000
        I researched docs for these patch.

        maria: https://mariadb.com/kb/en/mariadb/varchar/
        mysql: http://dev.mysql.com/doc/refman/5.7/en/char.html
        postgres: http://www.postgresql.org/docs/8.3/static/datatype-character.html
        derby: https://db.apache.org/derby/docs/10.7/ref/rrefsqlj41207.html
        oracle: https://docs.oracle.com/cd/B28359_01/server.111/b28320/limits001.htm#i287903
        (here varchar2 limit is just 4000)

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/charsyam/tajo feature/TAJO-1360

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/413.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #413


        commit 031d6e6f4d023ed9e014635d718239bf0291f625
        Author: clark.kang <clark.kang@kakao.com>
        Date: 2015-03-11T15:48:45Z

        TAJO-1360


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user charsyam opened a pull request: https://github.com/apache/tajo/pull/413 TAJO-1360 : VALUES_ field in OPTIONS table of catalog store should be longer. I suggest OPTIONS size as 4000, because max size of oracle varchar2 is just 4000 I researched docs for these patch. maria: https://mariadb.com/kb/en/mariadb/varchar/ mysql: http://dev.mysql.com/doc/refman/5.7/en/char.html postgres: http://www.postgresql.org/docs/8.3/static/datatype-character.html derby: https://db.apache.org/derby/docs/10.7/ref/rrefsqlj41207.html oracle: https://docs.oracle.com/cd/B28359_01/server.111/b28320/limits001.htm#i287903 (here varchar2 limit is just 4000) You can merge this pull request into a Git repository by running: $ git pull https://github.com/charsyam/tajo feature/ TAJO-1360 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/413.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #413 commit 031d6e6f4d023ed9e014635d718239bf0291f625 Author: clark.kang <clark.kang@kakao.com> Date: 2015-03-11T15:48:45Z TAJO-1360

          People

          • Assignee:
            charsyam DaeMyung Kang
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development