[CARBONDATA-558] Load performance bad when use_kettle=false

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-incubating
• Assignee: Gin-zhj
• Reporter: Gin-zhj
• Created: 23/Dec/16 09:17
• Updated: 29/Dec/16 08:07
• Resolved: 29/Dec/16 08:07

Description

When you import a data file, the measure column contains many empty strings， if use_kettle=false, the load performance has a sharp decline

I checked the logs of executor, many warnnings printed like below:
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-6 Cant not convert : to Numeric type value. Value considered as null.
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-1 Cant not convert : to Numeric type value. Value considered as null.
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-6 Cant not convert : to Numeric type value. Value considered as null.
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-1 Cant not convert : to Numeric type value. Value considered as null.
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-2 Cant not convert : to Numeric type value. Value considered as null.
16/12/22 07:03:12 WARN MeasureFieldConverterImpl: pool-22-thread-3 Cant not convert : to Numeric type value. Value considered as null.

Issue Links

• links to: GitHub Pull Request #459