[CARBONDATA-3881] concurrent main table compaction and SI load issue

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Assignee: Ajantha Bhat
• Reporter: Ajantha Bhat
• Created: 30/Jun/20 07:12
• Updated: 12/Oct/20 07:13
• Resolved: 12/Oct/20 07:13

Description

problem: consider a scenario, where segmentX has loaded to main table but failed to load to SI table. So, while loading another segmentY, we reload failed SI segmentX. this time if the segmentX is compacted in main table and clean files executed on it. SI load will fail and segmented will not be found in segmentMap of SI and it throws exception.

 

solution: just before reloading the failed SI segment. check if it is valid segment in main table.