[CARBONDATA-2953] Dataload fails when sort column is given, and query returns null value from another session

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Assignee: Akash R Nilugal
• Reporter: Akash R Nilugal
• Created: 20/Sep/18 13:02
• Updated: 21/Sep/18 13:19
• Resolved: 21/Sep/18 13:19

Description

1. when dataload is done with sort columns, it fails with following exeptions

java.lang.ClassCastException: java.lang.Integer cannot be cast to [B
at org.apache.carbondata.processing.sort.sortdata.IntermediateSortTempRowComparator.compare(IntermediateSortTempRowComparator.java:71)
at org.apache.carbondata.processing.loading.sort.unsafe.holder.UnsafeInmemoryHolder.compareTo(UnsafeInmemoryHolder.java:71)
at org.apache.carbondata.processing.loading.sort.unsafe.holder.UnsafeInmemoryHolder.compareTo(UnsafeInmemoryHolder.java:26)
at java.util.PriorityQueue.siftUpComparable(PriorityQueue.java:656)
at java.util.PriorityQueue.siftUp(PriorityQueue.java:647)
at java.util.PriorityQueue.offer(PriorityQueue.java:344)
at java.util.PriorityQueue.add(PriorityQueue.java:321)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeSingleThreadFinalSortFilesMerger.startSorting(UnsafeSingleThreadFinalSortFilesMerger.java:129)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeSingleThreadFinalSortFilesMerger.startFinalMerge(UnsafeSingleThreadFinalSortFilesMerger.java:94)
at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:110)
at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:55)
at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:112)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>(NewCarbonDataLoadRDD.scala:212)
at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD.internalCompute(NewCarbonDataLoadRDD.scala:188)
at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:78)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)

1. when two sessions are running in parallel, the follow below steps in session1
   1. drop table
   2. create table
   3. load data to table
2. follow below step in session2
   1. query on table(select * from table limit 1), then the query returns null result instead for proper result

Issue Links

• links to: GitHub Pull Request #2743