Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2822

Carbon Configuration - "carbon.invisible.segments.preserve.count" configuration property is not working as expected.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core, file-format
    • Labels:
      None
    • Environment:
      3 Node ANT cluster.

      Description

      For the carbon.invisible.segments.preserve.count configuration, it is not working as expected.

      Steps to reproduce:

      1) Setting up "carbon.invisible.segments.preserve.count=20" in carbon.properties and restarting the thrift server.

       

      2) After performing Loading 40 times and Compaction 4 times.

      3) Perform clean files, so that the tablestatus.history file would be generated with invisible segments details.

       So Total 44 segments would be created including visible and invisible segments.(40 load segment (like segment ID from 0,1,2...39) + 4 compacted new segment(like 0.1,20.1,22.1,0.2))

      In that, 41 segments information are present in the "tablestatus.history" file(*which holds invisible(marked for delete and compacted) segments details) and 3 segments information are present in the "tablestatus" file(which holds visible segments(0 .2 -final compacted segment) along with (1^st^ segment - 0th segment) and (last segment-39th segment)). *But invisible segment preserve count is configured to 20, which is not followed for the tablestatus.history file.

      Expected result:

      tablestatus.history file should preserve only the latest 20 segments, as per the configuration.

      Actual result:

      tablestatus.history file is having 41 invisible segments details.(which is above the configured value: 20)

       

      This is tested with ANT cluster.

        Attachments

        1. configuration.png
          76 kB
          Prasanna Ravichandran

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Prasanna Ravichandran Prasanna Ravichandran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: