Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21098

Set lineseparator csv multiline and csv write to \n

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Not A Problem
    • 2.2.2
    • None
    • Input/Output, SQL
    • None

    Description

      The Univocity-parser library uses the system line ending character as the default line ending character. Rather than remain dependent on the setting in this lib, we could set the default to \n. We cannot make this configurable for reading as it depends on LineReader from Hadoop, which has a hardcoded \n as line ending.

      Attachments

        Issue Links

          is related to

          Umbrella - An overarching type made of sub-tasks SPARK-21289 Text based formats do not support custom end-of-line delimiters

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #18304 [Github] Pull Request #18304 (danielvdende)

          Web Link GitHub Pull Request #18304

          Activity

            People

              Unassigned Unassigned
              danielvdende Daniel van der Ende
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: