[CARBONDATA-2258] Separate visible and invisible segments info into two files to reduce the size of tablestatus file.

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.0, 2.0.2
• Fix Version/s: 1.4.0
• Component/s: core
• Assignee: Zhichao Zhang
• Reporter: Zhichao Zhang
• Created: 15/Mar/18 09:13
• Updated: 23/Nov/20 06:07
• Resolved: 24/Mar/18 16:02

Description

The size of the tablestatus file is getting larger, there are many places will scan this file and it will impact the performance of reading this file.
According to the discussion on thread, it can append the
invisible segment list to the file called 'tablestatus.history' when execute
command 'CLEAN FILES FOR TABLE' (in method 'SegmentStatusManager.deleteLoadsAndUpdateMetadata') every time, separate visible and invisible segments into two files(tablestatus file and tablestatus.history file).
If later it needs to support listing all segments(include visible and invisible) list when execute 'SHOW SEGMENTS FOR TABLE', it just need to read from two files.

Issue Links

• links to: GitHub Pull Request #2091