[CARBONDATA-1324] generate global dictionary failed

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: data-load
• Environment: spark 1.6.2; carbondata 1.1.0
• Assignee: Unassigned
• Reporter: lingya
• Created: 20/Jul/17 15:26
• Updated: 02/Jun/20 03:51

Description

Recently I try to load data into carbodata with csv files,one of the lines is mismatch,All lines after this line are considered one line.

can you avoid this mismatch lines, leave the data behind continue load data into cardata?

As follows is error message:

java.lang.Exception: generate global dictionary failed, Error processing input: Length of parsed input (100001) exceeds the maximum number of characters defined in your parser settings (100000). Identified line separator characters in the parsed content. This may be the cause of the error. The line separator in your parser settings is set to '\n'.