Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-546

The timestamps are getting duplicated millis after ORC-306.

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • Java
    • None

    Description

      The Hive's TimestampColumnVector has a bad design with millis from 1970 and nanos within the second. This is consistent with java.sql.Timestamp, but it causes the millis to overlap with the nanos.

      ORC-306 changed the behavior from:
      millis: xxxx000, nanos: 123456789

      to:
      millis: xxxx123, nanos: 123456789

      that means that addition of the millis & nanos doubles the contribution of the millis.

      Attachments

        Issue Links

        causes

        Bug - A problem which impairs or prevents the functions of the product. SPARK-27594 spark.sql.orc.enableVectorizedReader causes milliseconds in Timestamp to be read incorrectly

        • Major - Major loss of function.
        • Resolved
        is caused by

        Test - A new unit, integration or system test. ORC-306 Fix incorrect workaround for bug in java.sql.Timestamp

        • Major - Major loss of function.
        • Closed
        links to

        Web Link GitHub Pull Request #420

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            omalley Owen O'Malley
            omalley Owen O'Malley
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment