[SPARK-26804] Spark sql carries newline char from last csv column when imported

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.0
• Component/s: Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Raj
• Created: 31/Jan/19 21:22
• Updated: 12/Dec/22 18:11
• Resolved: 25/May/21 01:42

Description

I am trying to generate external sql tables in DataBricks using Spark sql query. Below is my query. The query reads csv file and creates external table but it carries the newline char while creating the last column. Is there a way to resolve this issue? 

 

%sql
create table if not exists <<My table name>>
using CSV
options ("header"="true", "inferschema"="true","multiLine"="true", "escape"='"')
location <my csv path>