[CARBONDATA-2413] After running CarbonWriter, there is null directory about datamap

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Assignee: Bo Xu
• Reporter: Bo Xu
• Created: 28/Apr/18 01:28
• Updated: 19/May/18 02:45
• Resolved: 19/May/18 02:45

Description

After running CarbonWriter, there is null directory:

root@ecs-909c:/huawei/xubo/git/carbondata2/null/_system# ls
datamap.mdtfile
root@ecs-909c:/huawei/xubo/git/carbondata2/null/_system# cd ../../
root@ecs-909c:/huawei/xubo/git/carbondata2# git status
On branch CARBONDATA-2392-CarbonReaderClose
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        null/

Another error:

java.io.IOException: No such file or directory
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1012)
	at org.apache.carbondata.core.datastore.filesystem.LocalCarbonFile.createNewFile(LocalCarbonFile.java:416)
	at org.apache.carbondata.core.datastore.impl.FileFactory.createNewFile(FileFactory.java:221)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.touchMDTFile(DiskBasedDMSchemaStorageProvider.java:198)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.checkAndReloadDataMapSchemas(DiskBasedDMSchemaStorageProvider.java:186)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.retrieveSchemas(DiskBasedDMSchemaStorageProvider.java:106)
	at org.apache.carbondata.core.datamap.DataMapStoreManager.getDataMapSchemasOfTable(DataMapStoreManager.java:121)
	at org.apache.carbondata.core.datamap.DataMapStoreManager.getAllDataMap(DataMapStoreManager.java:101)
	at org.apache.carbondata.processing.datamap.DataMapWriterListener.registerAllWriter(DataMapWriterListener.java:57)
	at org.apache.carbondata.processing.store.CarbonFactDataHandlerModel.createCarbonFactDataHandlerModel(CarbonFactDataHandlerModel.java:264)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.processRange(DataWriterProcessorStepImpl.java:166)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.access$100(DataWriterProcessorStepImpl.java:53)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl$WriterForwarder.call(DataWriterProcessorStepImpl.java:156)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl$WriterForwarder.call(DataWriterProcessorStepImpl.java:141)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
2018-05-04 10:23:57,353 ERROR [org.apache.carbondata.common.logging.impl.StandardLogService.logErrorMessage(StandardLogService.java:143)] CarbonRecordWriter:_tempTable Failed for table: _tempTable in DataWriterProcessorStepImpl
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: 
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:121)
	at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
	at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$1.run(CarbonTableOutputFormat.java:251)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: No such file or directory
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:118)
	... 7 more
Caused by: java.lang.RuntimeException: java.io.IOException: No such file or directory
	at org.apache.carbondata.processing.datamap.DataMapWriterListener.registerAllWriter(DataMapWriterListener.java:60)
	at org.apache.carbondata.processing.store.CarbonFactDataHandlerModel.createCarbonFactDataHandlerModel(CarbonFactDataHandlerModel.java:264)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.processRange(DataWriterProcessorStepImpl.java:166)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.access$100(DataWriterProcessorStepImpl.java:53)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl$WriterForwarder.call(DataWriterProcessorStepImpl.java:156)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl$WriterForwarder.call(DataWriterProcessorStepImpl.java:141)
	... 4 more
Caused by: java.io.IOException: No such file or directory
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1012)
	at org.apache.carbondata.core.datastore.filesystem.LocalCarbonFile.createNewFile(LocalCarbonFile.java:416)
	at org.apache.carbondata.core.datastore.impl.FileFactory.createNewFile(FileFactory.java:221)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.touchMDTFile(DiskBasedDMSchemaStorageProvider.java:198)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.checkAndReloadDataMapSchemas(DiskBasedDMSchemaStorageProvider.java:186)
	at org.apache.carbondata.core.metadata.schema.table.DiskBasedDMSchemaStorageProvider.retrieveSchemas(DiskBasedDMSchemaStorageProvider.java:106)
	at org.apache.carbondata.core.datamap.DataMapStoreManager.getDataMapSchemasOfTable(DataMapStoreManager.java:121)
	at org.apache.carbondata.core.datamap.DataMapStoreManager.getAllDataMap(DataMapStoreManager.java:101)
	at org.apache.carbondata.processing.datamap.DataMapWriterListener.registerAllWriter(DataMapWriterListener.java:57)
	... 9 more

Issue Links

• links to: GitHub Pull Request #2246