Hive
  1. Hive
  2. HIVE-2803

utc_from_timestamp and utc_to_timestamp returns incorrect results.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.10.0
    • Component/s: UDF
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      How to reproduce:

      $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt
      hive> create table ts1(t1 timestamp);
      hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1;
      hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST') from ts1 limit 1;
      

      The following result is expected:

       2011-12-25 09:00:00.123456      2011-12-25 18:00:00.123456      2011-12-25 18:00:00.123456
      

      However, the above query return incorrect result like this:

       2011-12-26 03:00:00.492456      2011-12-26 03:00:00.492456      2011-12-26 03:00:00.492456
      

      This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime() improperly.
      On evaluating query, timestamp argument always returns the same instance.
      GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance.
      That means it adds all offsets in the query.

      1. HIVE-2803.D1959.1.patch
        5 kB
        tamtam180
      2. ASF.LICENSE.NOT.GRANTED--HIVE-2803.D1959.1.patch
        5 kB
        Phabricator
      3. HIVE-2803.1.patch.txt
        6 kB
        tamtam180

        Activity

        Hide
        tamtam180 added a comment -

        Attached is the patch which fixes this issue.
        It changed so that applyOffset() return a new instance.

        Show
        tamtam180 added a comment - Attached is the patch which fixes this issue. It changed so that applyOffset() return a new instance.
        Hide
        Carl Steinbach added a comment -

        @tamtam180: If this patch is ready for review can you please change the status to 'Patch Available' and submit a review request using Phabricator? Instructions are located here:
        https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ReviewProcess

        Thanks!

        Show
        Carl Steinbach added a comment - @tamtam180: If this patch is ready for review can you please change the status to 'Patch Available' and submit a review request using Phabricator? Instructions are located here: https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ReviewProcess Thanks!
        Hide
        Phabricator added a comment -

        tamtam180 requested code review of "HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results.".
        Reviewers: JIRA

        https://issues.apache.org/jira/browse/HIVE-2803

        HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results.

        It changed so that applyOffset() return a new instance.

        How to reproduce:

        $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt
        hive> create table ts1(t1 timestamp);
        hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1;
        hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST') from ts1 limit 1;

        The following result is expected:

        2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25 18:00:00.123456

        However, the above query return incorrect result like this:

        2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456

        This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime() improperly.
        On evaluating query, timestamp argument always returns the same instance.
        GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance.
        That means it adds all offsets in the query.

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D1959

        AFFECTED FILES
        ql/src/test/results/clientpositive/timestamp_udf.q.out
        ql/src/test/queries/clientpositive/timestamp_udf.q
        ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/4143/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - tamtam180 requested code review of " HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results.". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2803 HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results. It changed so that applyOffset() return a new instance. How to reproduce: $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt hive> create table ts1(t1 timestamp); hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1; hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST') from ts1 limit 1; The following result is expected: 2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25 18:00:00.123456 However, the above query return incorrect result like this: 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime() improperly. On evaluating query, timestamp argument always returns the same instance. GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance. That means it adds all offsets in the query. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1959 AFFECTED FILES ql/src/test/results/clientpositive/timestamp_udf.q.out ql/src/test/queries/clientpositive/timestamp_udf.q ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4143/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        tamtam180 added a comment -

        Carl,
        thanks your advice.
        I tried use of Phabricator, and changed status to 'Patch Available'.

        Show
        tamtam180 added a comment - Carl, thanks your advice. I tried use of Phabricator, and changed status to 'Patch Available'.
        Hide
        Carl Steinbach added a comment -

        Committed to trunk. Thanks Tamtam180!

        Show
        Carl Steinbach added a comment - Committed to trunk. Thanks Tamtam180!
        Hide
        Carl Steinbach added a comment -

        @tamtam180: can you please upload another copy of the patch and click the box that grants license rights to the ASF? Thanks!

        Show
        Carl Steinbach added a comment - @tamtam180: can you please upload another copy of the patch and click the box that grants license rights to the ASF? Thanks!
        Hide
        Phabricator added a comment -

        tamtam180 has committed the revision "HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results.".

        Change committed by cws.

        REVISION DETAIL
        https://reviews.facebook.net/D1959

        COMMIT
        https://reviews.facebook.net/rHIVE1329507

        Show
        Phabricator added a comment - tamtam180 has committed the revision " HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results.". Change committed by cws. REVISION DETAIL https://reviews.facebook.net/D1959 COMMIT https://reviews.facebook.net/rHIVE1329507
        Hide
        tamtam180 added a comment -

        Final version which grants rights to the ASF.

        Show
        tamtam180 added a comment - Final version which grants rights to the ASF.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1392 (See https://builds.apache.org/job/Hive-trunk-h0.21/1392/)
        HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect
        results.
        (Kiyotaka Suzuki via Carl Steinbach)

        Summary:
        https://issues.apache.org/jira/browse/HIVE-2803

        HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results.

        It changed so that applyOffset() return a new instance.

        How to reproduce:

        $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt
        hive> create table ts1(t1 timestamp);
        hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1;
        hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST')
        from ts1 limit 1;

        The following result is expected:

        2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25
        18:00:00.123456

        However, the above query return incorrect result like this:

        2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26
        03:00:00.492456

        This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime()
        improperly.
        On evaluating query, timestamp argument always returns the same instance.
        GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance.
        That means it adds all offsets in the query.

        Test Plan: EMPTY

        Reviewers: JIRA

        Differential Revision: https://reviews.facebook.net/D1959 (Revision 1329507)

        Result = FAILURE
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329507
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java
        • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q
        • /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1392 (See https://builds.apache.org/job/Hive-trunk-h0.21/1392/ ) HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results. (Kiyotaka Suzuki via Carl Steinbach) Summary: https://issues.apache.org/jira/browse/HIVE-2803 HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results. It changed so that applyOffset() return a new instance. How to reproduce: $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt hive> create table ts1(t1 timestamp); hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1; hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST') from ts1 limit 1; The following result is expected: 2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25 18:00:00.123456 However, the above query return incorrect result like this: 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime() improperly. On evaluating query, timestamp argument always returns the same instance. GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance. That means it adds all offsets in the query. Test Plan: EMPTY Reviewers: JIRA Differential Revision: https://reviews.facebook.net/D1959 (Revision 1329507) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329507 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect
        results.
        (Kiyotaka Suzuki via Carl Steinbach)

        Summary:
        https://issues.apache.org/jira/browse/HIVE-2803

        HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results.

        It changed so that applyOffset() return a new instance.

        How to reproduce:

        $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt
        hive> create table ts1(t1 timestamp);
        hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1;
        hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST')
        from ts1 limit 1;

        The following result is expected:

        2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25
        18:00:00.123456

        However, the above query return incorrect result like this:

        2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26
        03:00:00.492456

        This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime()
        improperly.
        On evaluating query, timestamp argument always returns the same instance.
        GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance.
        That means it adds all offsets in the query.

        Test Plan: EMPTY

        Reviewers: JIRA

        Differential Revision: https://reviews.facebook.net/D1959 (Revision 1329507)

        Result = ABORTED
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329507
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java
        • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q
        • /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect results. (Kiyotaka Suzuki via Carl Steinbach) Summary: https://issues.apache.org/jira/browse/HIVE-2803 HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results. It changed so that applyOffset() return a new instance. How to reproduce: $ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt hive> create table ts1(t1 timestamp); hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1; hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST') from ts1 limit 1; The following result is expected: 2011-12-25 09:00:00.123456 2011-12-25 18:00:00.123456 2011-12-25 18:00:00.123456 However, the above query return incorrect result like this: 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 2011-12-26 03:00:00.492456 This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime() improperly. On evaluating query, timestamp argument always returns the same instance. GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance. That means it adds all offsets in the query. Test Plan: EMPTY Reviewers: JIRA Differential Revision: https://reviews.facebook.net/D1959 (Revision 1329507) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329507 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out
        Hide
        Ashutosh Chauhan added a comment -

        This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

        Show
        Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

          People

          • Assignee:
            tamtam180
            Reporter:
            tamtam180
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development