Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4373

Drill and Hive have incompatible timestamp representations in parquet

    XMLWordPrintableJSON

    Details

      Description

      git.commit.id.abbrev=83d460c

      I created a parquet file with a timestamp type using Drill. Now if I define a hive table on top of the parquet file and use "timestamp" as the column type, drill fails to read the hive table through the hive storage plugin

      Implementation:

      Added int96 to timestamp converter for both parquet readers and controling it by system / session option "store.parquet.int96_as_timestamp".
      The value of the option is false by default for the proper work of the old query scripts with the "convert_from TIMESTAMP_IMPALA" function.

      When the option is true using of that function is unnesessary and can lead to the query fail.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vitalii Vitalii Diravka
                Reporter:
                rkins Rahul Challapalli
                Reviewer:
                Krystal
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: