Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12420 Have a built-in CSV data source implementation
  3. SPARK-13667

Support for specifying custom date format for date and timestamp types

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.0.0
    • SQL
    • None

    Description

      Currently, CSV data source does not support to parse date and timestamp types in custom format and infer the type of timestamp type in custom format.

      It looks quite many of users want this feature. It would be great to set custom date format.

      This was reported in spark-csv.
      https://github.com/databricks/spark-csv/issues/279
      https://github.com/databricks/spark-csv/issues/262
      https://github.com/databricks/spark-csv/issues/266

      Currently I submitted a PR for this in spark-csv
      https://github.com/databricks/spark-csv/pull/280

      Attachments

        Activity

          People

            hyukjin.kwon Hyukjin Kwon
            hyukjin.kwon Hyukjin Kwon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: