Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19723

Arrow serde: "Unsupported data type: Timestamp(NANOSECOND, null)"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0
    • None

    Description

      Spark's Arrow support only provides Timestamp at MICROSECOND granularity. Spark 2.3.0 won't accept NANOSECOND. Switch it back to MICROSECOND.
      The unit test org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow will just need to change the assertion to test microsecond. And we'll need to add this to documentation on supported datatypes.

      Attachments

        1. HIVE-19723.1.patch
          11 kB
          Teddy Choi
        2. HIVE-19732.2.patch
          13 kB
          Teddy Choi
        3. HIVE-19723.3.patch
          15 kB
          Teddy Choi
        4. HIVE-19723.4.patch
          22 kB
          Teddy Choi

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-19853 Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #360

          Web Link GitHub Pull Request #369

          Activity

            People

              teddy.choi Teddy Choi
              teddy.choi Teddy Choi
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: