Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4373

Drill and Hive have incompatible timestamp representations in parquet

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      git.commit.id.abbrev=83d460c

      I created a parquet file with a timestamp type using Drill. Now if I define a hive table on top of the parquet file and use "timestamp" as the column type, drill fails to read the hive table through the hive storage plugin

      Implementation:

      Added int96 to timestamp converter for both parquet readers and controling it by system / session option "store.parquet.int96_as_timestamp".
      The value of the option is false by default for the proper work of the old query scripts with the "convert_from TIMESTAMP_IMPALA" function.

      When the option is true using of that function is unnesessary and can lead to the query fail.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              vitalii Vitalii Diravka
              Reporter:
              rkins Rahul Kumar Challapalli
              Reviewer:
              Krystal

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment