Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1741

Two tables having same time zone display different timestamps

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: Storage
    • Labels:

      Description

      Create two tables with same scheme. Note 'GMT+9'.
      (Table names are timezone and timezone_text)

      CREATE EXTERNAL TABLE timezone 
      (t_timestamp TIMESTAMP, 
      t_date DATE) 
      USING TEXT WITH ('timezone'='GMT+9') 
      LOCATION 'hdfs://localhost:8020/timezone';
      

      Timezone table already has data.

      1980-4-1 01:50:30.010|1980-04-01
      80/4/1 1:50:30 AM|80/4/1
      1980 April 1 1:50:30|1980-04-01
      

      Then, copy data to timezone_text.

      INSERT INTO timezone_text SELECT * FROM timezone;
      

      Check timezone_text with some editor. It looks ok.

      1980-04-01 01:50:30.01+09|1980-04-01
      1980-04-01 01:50:30+09|1980-04-01
      1980-04-01 01:50:30+09|1980-04-01
      

      But in tsql, two tables show different time values.

      select * from timezone
      t_timestamp,  t_date
      -------------------------------
      1980-04-01 01:50:30.01,  1980-04-01
      1980-04-01 01:50:30,  1980-04-01
      1980-04-01 01:50:30,  1980-04-01
      
      select * from timezone_text
      t_timestamp,  t_date
      -------------------------------
      1980-03-31 16:50:30.01,  1980-04-01
      1980-03-31 16:50:30,  1980-04-01
      1980-03-31 16:50:30,  1980-04-01
      
      1. TAJO-1741.patch
        1 kB
        Jongyoung Park

        Issue Links

          Activity

          Hide
          eminency Jongyoung Park added a comment -

          Data in 'timezone_text' have timezone itself like that:

          1980-04-01 01:50:30.01+09
          

          So it looks timezone is processed twice - timezone of TimeMeta(record timezone) and timezone of table option.

          Show
          eminency Jongyoung Park added a comment - Data in 'timezone_text' have timezone itself like that: 1980-04-01 01:50:30.01+09 So it looks timezone is processed twice - timezone of TimeMeta(record timezone) and timezone of table option.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user eminency opened a pull request:

          https://github.com/apache/tajo/pull/699

          TAJO-1741: Two tables having same time zone display different timestamps

          includeTimezone flag is for TIMESTAMPZ type, so changed to 'false'

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/eminency/tajo TAJO-1741

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/699.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #699


          commit 192c1ef505749a0ef0372c83783abc6988f446ad
          Author: Jongyoung Park <eminency@gmail.com>
          Date: 2015-08-19T02:58:53Z

          includeTimezone flag is for TIMESTAMPZ type, so changed to 'false'


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user eminency opened a pull request: https://github.com/apache/tajo/pull/699 TAJO-1741 : Two tables having same time zone display different timestamps includeTimezone flag is for TIMESTAMPZ type, so changed to 'false' You can merge this pull request into a Git repository by running: $ git pull https://github.com/eminency/tajo TAJO-1741 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/699.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #699 commit 192c1ef505749a0ef0372c83783abc6988f446ad Author: Jongyoung Park <eminency@gmail.com> Date: 2015-08-19T02:58:53Z includeTimezone flag is for TIMESTAMPZ type, so changed to 'false'
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/699#issuecomment-132450837

          +1
          The change looks straightforward. I'll change your comments into a java doc format before committing.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/699#issuecomment-132450837 +1 The change looks straightforward. I'll change your comments into a java doc format before committing.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/699#issuecomment-132450929

          @eminency

          I missed one thing. Could you add insert tests using timezone?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/699#issuecomment-132450929 @eminency I missed one thing. Could you add insert tests using timezone?
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12751192/TAJO-1741.patch
          against master revision release-0.9.0-rc0-416-g6f28fa5.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-storage/tajo-storage-hdfs.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/827//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/827//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/827//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12751192/TAJO-1741.patch against master revision release-0.9.0-rc0-416-g6f28fa5. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/827//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/827//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/827//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user eminency commented on the pull request:

          https://github.com/apache/tajo/pull/699#issuecomment-132891279

          @hyunsik

          Thank you for review.
          I applied your suggestion right before.

          Show
          githubbot ASF GitHub Bot added a comment - Github user eminency commented on the pull request: https://github.com/apache/tajo/pull/699#issuecomment-132891279 @hyunsik Thank you for review. I applied your suggestion right before.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/699#issuecomment-132902970

          +1
          The patch looks good to me, and it looks enough for the unit test to verify this bug fix.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/699#issuecomment-132902970 +1 The patch looks good to me, and it looks enough for the unit test to verify this bug fix.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/699

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/699
          Hide
          hyunsik Hyunsik Choi added a comment -

          I just committed this patch to both branches, branch-0.11.0 and master. Thank you for your contribution.

          Show
          hyunsik Hyunsik Choi added a comment - I just committed this patch to both branches, branch-0.11.0 and master. Thank you for your contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #448 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/448/)
          TAJO-1741: Two tables having same time zone display different timestamps. (hyunsik: rev 2d2f192d78249d8acc812a1a3d08f5caceee9208)

          • tajo-core-tests/src/test/resources/queries/TestSelectQuery/datetime_table_timezoned_ddl2.sql
          • CHANGES
          • tajo-core-tests/src/test/resources/queries/TestSelectQuery/testLoadIntoTimezonedTable.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
          • tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #448 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/448/ ) TAJO-1741 : Two tables having same time zone display different timestamps. (hyunsik: rev 2d2f192d78249d8acc812a1a3d08f5caceee9208) tajo-core-tests/src/test/resources/queries/TestSelectQuery/datetime_table_timezoned_ddl2.sql CHANGES tajo-core-tests/src/test/resources/queries/TestSelectQuery/testLoadIntoTimezonedTable.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #809 (See https://builds.apache.org/job/Tajo-master-build/809/)
          TAJO-1741: Two tables having same time zone display different timestamps. (hyunsik: rev 2d2f192d78249d8acc812a1a3d08f5caceee9208)

          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java
          • CHANGES
          • tajo-core-tests/src/test/resources/queries/TestSelectQuery/datetime_table_timezoned_ddl2.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
          • tajo-core-tests/src/test/resources/queries/TestSelectQuery/testLoadIntoTimezonedTable.sql
          • tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #809 (See https://builds.apache.org/job/Tajo-master-build/809/ ) TAJO-1741 : Two tables having same time zone display different timestamps. (hyunsik: rev 2d2f192d78249d8acc812a1a3d08f5caceee9208) tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java CHANGES tajo-core-tests/src/test/resources/queries/TestSelectQuery/datetime_table_timezoned_ddl2.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java tajo-core-tests/src/test/resources/queries/TestSelectQuery/testLoadIntoTimezonedTable.sql tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java

            People

            • Assignee:
              eminency Jongyoung Park
              Reporter:
              eminency Jongyoung Park
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development