Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-8315

Exception when write null value to table with timestamp partitioning

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 1.1.0
    • hive, spark-sql
    • None

    Description

      Issue:

      Exception occurs when writing a null value to a partition table with a timestamp partition column.

      Expectation:

      Needs to support timestamp type partition column value as null similar to writing data to hive table.

      Code:

      spark.sql(
              s"""
                 |CREATE TABLE hudi_test_null_partition (
                 |  id INT,
                 |  boolean_field BOOLEAN,
                 |  float_field FLOAT,
                 |  byte_field BYTE,
                 |  short_field SHORT,
                 |  decimal_field DECIMAL(10, 5),
                 |  date_field DATE,
                 |  string_field STRING,
                 |  timestamp_field TIMESTAMP
                 |) USING hudi
                 | TBLPROPERTIES (primaryKey = 'id')
                 | PARTITIONED BY (boolean_field, float_field, byte_field, short_field, decimal_field, date_field, string_field, timestamp_field)
           """.stripMargin)spark.sql(
      s"""
         |INSERT INTO hudi_test_null_partition VALUES
         |(1, TRUE, CAST(1.0 as FLOAT), 1, 1, 1234.56789, DATE '2021-01-05', 'partition1', null),
         |(2, FALSE,CAST(2.0 as FLOAT), 2, 2, 6789.12345, DATE '2021-01-06', 'partition2', TIMESTAMP '2021-01-06 11:00:00')
      """.stripMargin)

      Reference Issue:

      https://github.com/apache/hudi/issues/11900

       

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            rangareddy.avula@gmail.com Ranga Reddy
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: