[CARBONDATA-4060] Launching a new spark job for the data file merge operation in SI segments.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.1
• Component/s: core, data-load
• Assignee: Unassigned
• Reporter: Karan
• Created: 27/Nov/20 08:09
• Updated: 27/Nov/20 08:09

Description

Currently, data files merge operation for SI segments is done for each segment sequentially. A separate spark job can be launched to improve the performance in case the main table have large number of segments.