[CARBONDATA-1337] Problem while intermediate merging

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0
• Component/s: data-load
• Assignee: Zhichao Zhang
• Reporter: Zhichao Zhang
• Created: 28/Jul/17 08:12
• Updated: 29/Jul/17 14:32
• Resolved: 29/Jul/17 01:35

Description

With PR-1198(CARBONDATA-1281), when loading data, there was error occuring, as following:

2017-07-28 13:10:49,443 - ERROR - org.apache.carbondata.common.logging.impl.StandardLogService.logErrorMessage(StandardLogService.java:143) - pool-17-thread-1 -pool-17-thread-1 Problem while intermediate merging
org.apache.carbondata.processing.sortandgroupby.exception.CarbonSortKeyAndGroupByException: Problem while getting the file
        at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.initialize(IntermediateFileMerger.java:168)
        at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.call(IntermediateFileMerger.java:110)
        at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.call(IntermediateFileMerger.java:37)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: [Ljava.lang.String;@54218780/carbondata_hundred_million_pr11987419814281939480.merge (No such file or directory)
        at java.io.FileOutputStream.open0(Native Method)
        at java.io.FileOutputStream.open(FileOutputStream.java:270)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
        at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.initialize(IntermediateFileMerger.java:163)
        ... 6 more

According to xuchuanyin's suggestion, it needs to fix the code of method SortIntermediateFileMerger.startIntermediateMerging.

Issue Links

• links to: GitHub Pull Request #1206