[CARBONDATA-4043] Fix data load failure issue for columns added in legacy store

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Assignee: Unassigned
• Reporter: Indhumathi Muthumurugesh
• Created: 23/Oct/20 11:49
• Updated: 03/Nov/20 14:14
• Resolved: 02/Nov/20 13:47

Description

When dimension is added in older versions like 1.1, by default it will be sort column. In sort step we assume data will be coming as sort column in the beginning. But the added column will be at last eventhough sort column. So, while building the dataload configurations for loading data, we rearrange the columns(dimensions and datafields) in order to bring the sort column to beginning and no-sort to last and revert them back to schema order before FinalMerge/DataWriter step.

Issue:
Data loading is failing because of castException in data writing step in case of NO_SORT and in final sort step in case of LOCAL_SORT.