[CARBONDATA-3820] Fix CDC failure when sort columns present in source dataframe

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Assignee: Unassigned
• Reporter: Xingjun Hao
• Created: 13/May/20 00:50
• Updated: 03/Jan/21 15:38
• Resolved: 17/Jun/20 08:50

Description

If there is GloabalSort table in the CDC Flow. The following exception will
be throwed:

Exception in thread "main" java.lang.RuntimeException: column: id specified
in sort columns does not exist in schema
        at
org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildTableSchema(CarbonWriterBuilder.java:828)
        at
org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildCarbonTable(CarbonWriterBuilder.java:794)
        at
org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildLoadModel(CarbonWriterBuilder.java:720)
        at
org.apache.spark.sql.carbondata.execution.datasources.CarbonSparkDataSourceUtil$.prepareLoadModel(CarbonSparkDataSourceUtil.scala:281)
        at
org.apache.spark.sql.carbondata.execution.datasources.SparkCarbonFileFormat.prepareWrite(SparkCarbonFileFormat.scala:141)
        at
org.apache.spark.sql.execution.command.mutation.merge.CarbonMergeDataSetCommand.processIUD(CarbonMergeDataSetCommand.scala:269)
        at
org.apache.spark.sql.execution.command.mutation.merge.CarbonMergeDataSetCommand.processData(CarbonMergeDataSetCommand.scala:152)