[ASTERIXDB-2250] Component files are not cleaned up during failed FLUSH/MERGE

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: STO - Storage
• Assignee: Chen Luo
• Reporter: Chen Luo
• Created: 17/Jan/18 17:26
• Updated: 03/Feb/18 18:27
• Resolved: 03/Feb/18 18:27

Description

When a flush/merge fails, the component bulkloader (or in general the flush/merge operation) didn't clean up the component files. As a result, the next time when the flush/merge is scheduled again, the system will complain the files already exist on disk.

org.apache.hyracks.api.exceptions.HyracksDataException: HYR0082: Failed to create the file /Users/luochen/Documents/Research/projects/asterixdb/asterixdb/asterix-app/target/tmp/asterix_nc1/iodevice1/storage/partition_0/test/LineItem/0/idx_LineItem_partkey/2018-01-17-09-19-03-524_2018-01-17-09-19-03-510_b because it already exists

at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:53) ~[classes/:?]

at org.apache.hyracks.api.util.IoUtil.create(IoUtil.java:90) ~[classes/:?]

at org.apache.hyracks.storage.common.buffercache.BufferCache.createFile(BufferCache.java:825) ~[classes/:?]

at org.apache.hyracks.storage.am.common.impls.AbstractTreeIndex.create(AbstractTreeIndex.java:83) ~[classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMDiskComponent.activate(AbstractLSMDiskComponent.java:155) ~[classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.createDiskComponent(AbstractLSMIndex.java:461) ~[classes/:?]

at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.doMerge(LSMBTree.java:355) ~[classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:744) ~[classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:628) [classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:127) [classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.callInternal(MergeOperation.java:56) [classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.AbstractIoOperation.call(AbstractIoOperation.java:55) [classes/:?]

at org.apache.hyracks.storage.am.lsm.common.impls.AbstractIoOperation.call(AbstractIoOperation.java:31) [classes/:?]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]