[CARBONDATA-2822] Carbon Configuration - "carbon.invisible.segments.preserve.count" configuration property is not working as expected.

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Invalid
• Component/s: core, file-format
• Environment: 3 Node ANT cluster.
• Assignee: Unassigned
• Reporter: Prasanna Ravichandran
• Created: 03/Aug/18 11:38
• Updated: 07/Aug/18 10:50
• Resolved: 07/Aug/18 10:49

Description

For the carbon.invisible.segments.preserve.count configuration, it is not working as expected.

Steps to reproduce:

1) Setting up "carbon.invisible.segments.preserve.count=20" in carbon.properties and restarting the thrift server.

 

2) After performing Loading 40 times and Compaction 4 times.

3) Perform clean files, so that the tablestatus.history file would be generated with invisible segments details.

 So Total 44 segments would be created including visible and invisible segments.(40 load segment (like segment ID from 0,1,2...39) + 4 compacted new segment(like 0.1,20.1,22.1,0.2))

In that, 41 segments information are present in the "tablestatus.history" file(*which holds invisible(marked for delete and compacted) segments details) and 3 segments information are present in the "tablestatus" file(which holds visible segments(0 .2 -final compacted segment) along with (1^st^ segment - 0th segment) and (last segment-39th segment)). *But invisible segment preserve count is configured to 20, which is not followed for the tablestatus.history file.

Expected result:

tablestatus.history file should preserve only the latest 20 segments, as per the configuration.

Actual result:

tablestatus.history file is having 41 invisible segments details.(which is above the configured value: 20)

 

This is tested with ANT cluster.