Description
【Test step】:
1. Create a table with 'sort_column' tblproperty.
2. Load data into that table with the 'SORT_COLUMN_BOUNDS' property.
Test queries:
drop table if exists sortcolumnboundtable1;
CREATE TABLE sortcolumnboundtable1 (ID Int, date Timestamp, country String, name String, phonetype String, serialname String, salary Int) STORED BY 'carbondata' tblproperties('sort_columns'='ID,name');
LOAD DATA INPATH '/user/prasanna/source_for_sort_column_bounds.csv' INTO TABLE sortcolumnboundtable1 OPTIONS('fileheader'='ID,date,country,name,phonetype,serialname,salary','sort_column_bounds'='10,aab1;20,aab1;100,aab1;1000,aab1','timestampformat'='yyyy/MM/dd');
【check the log】
8/06/25 16:14:32 INFO loading.DataLoadExecutor: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] Data Loading is started for table sortcolumnboundtable1 18/06/25 16:14:32 ERROR impl.UnsafeParallelReadMergeSorterWithColumnRangeImpl: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] set temp location: /tmp/carbon2916995416352957_0/Fact/Part0/Segment_0/0/sortrowtmp 18/06/25 16:14:32 INFO memory.UnsafeSortMemoryManager: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] It is not recommended to keep unsafe memory size less than 1024MB, so setting default value to 1024 18/06/25 16:14:32 INFO memory.UnsafeSortMemoryManager: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] Sort Memory manager is created with size 1073741824 with org.apache.carbondata.core.memory.UnsafeMemoryAllocator@cd24739 18/06/25 16:14:32 INFO memory.UnsafeMemoryManager: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] Working Memory manager is created with size 536870912 with org.apache.carbondata.core.memory.UnsafeMemoryAllocator@cd24739 18/06/25 16:14:32 ERROR impl.UnsafeParallelReadMergeSorterWithColumnRangeImpl: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] set temp location: /tmp/carbon2916995416352957_0/Fact/Part0/Segment_0/0/sortrowtmp 18/06/25 16:14:32 ERROR impl.UnsafeParallelReadMergeSorterWithColumnRangeImpl: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] set temp location: /tmp/carbon2916995416352957_0/Fact/Part0/Segment_0/0/sortrowtmp 18/06/25 16:14:32 ERROR impl.UnsafeParallelReadMergeSorterWithColumnRangeImpl: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] set temp location: /tmp/carbon2916995416352957_0/Fact/Part0/Segment_0/0/sortrowtmp 18/06/25 16:14:32 ERROR impl.UnsafeParallelReadMergeSorterWithColumnRangeImpl: [Executor task launch worker for task 0][partitionID:sortcolumnboundtable1;queryID:2916995336294991] set temp location: /tmp/carbon2916995416352957_0/Fact/Part0/Segment_0/0/sortrowtmp
Attachments
Issue Links
- links to