Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
-
None
Description
Carbon uses tablestatus file to record segment status and details of each segment during each load. This tablestatus enables carbon to support concurrent loads and reads without data inconsistency or corruption.
So it is very important feature of carbondata and we should have clean interfaces to maintain it. Current tablestatus updation is shattered to multiple places and there is no clean interface, so I am proposing to refactor current SegmentStatusManager interface and bringing all tablestatus operations to single interface.
This new interface allows to add table status to any other storage like DB. This is needed for S3 type object stores as these are eventually consistent.