Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3555

re-use spark config for parquet timestamp format instead of having our own config

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • spark

    Description

      We have two diff configs to set the right timestamp format. 
      "hoodie.parquet.outputtimestamptype": "TIMESTAMP_MICROS",
      and spark config
      --conf spark.sql.parquet.outputTimestampType=TIMESTAMP_MICROS
       

      We should deprecate our own config and just rely on spark's configs. 

       

      Attachments

        Activity

          People

            liujinhui liujinhui
            shivnarayan sivabalan narayanan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: