Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16216

CSV data source does not write date and timestamp correctly

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.0.0
    • 2.0.1, 2.1.0
    • SQL

    Description

      Currently, CSV data source write DateType and TimestampType as below:

      +----------------+
      |            date|
      +----------------+
      |1440637200000000|
      |1414459800000000|
      |1454040000000000|
      +----------------+
      

      It would be nicer if it write dates and timestamps as a formatted string just like JSON data sources.

      Also, CSV data source currently supports dateFormat option to read dates and timestamps in a custom format. It might be better if this option can be applied in writing as well.

      Attachments

        Issue Links

          Activity

            People

              hyukjin.kwon Hyukjin Kwon
              hyukjin.kwon Hyukjin Kwon
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: