[HIVE-23058] Compaction task reattempt fails with FileAlreadyExistsException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Hive
• Assignee: Riju Trivedi
• Reporter: Riju Trivedi
• Created: 20/Mar/20 14:45
• Updated: 17/Nov/22 08:50
• Resolved: 14/Apr/20 16:55

Description

Issue occurs when compaction attempt is relaunched after first task attempt failure due to preemption by Scheduler or any other reason.

Since _tmp directory was created by first attempt and was left uncleaned after task attempt failure. Second attempt of the the task fails with "FileAlreadyExistsException" exception.

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.FileAlreadyExistsException): /warehouse/tablespace/managed/hive/default.db/compaction_test/_tmp_3670bbef-ba7a-4c10-918d-9a2ee17cbd22/base_0000186/bucket_00005 for client 10.xx.xx.xxx already exists

Issue Links

• is cloned by: HIVE-18044 CompactorMR.CompactorOutputCommitter.abortTask() not implemented
• is related to: HIVE-24882 Compaction task reattempt fails with FileAlreadyExistsException for DeleteEventWriter