Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Writer Core
    • Labels:
      None

      Description

      For now, metadata table has inline compaction. But we need to come up with a strategy to support async compaction. 

      Since MDT compaction is fenced based on inflight requests in datatable, if for some reason, compaction in data table kept on failing and never succeeds, we will never compact metadata table as well. This might turn out to be detrimental. 

      So, we should come up with a strategy to support async compaction in metadata table. 

       

      Some nuances: 

      If there are delta commits corresponding to rollback, we should ensure the final state in base table reflect that and don't miss out on any details. 

      F1 added with dc1. f1 removed by dc3 (rollback in datatable) and again f2 (added by same commit instant) when retried in datatable (compaction for eg). 

      So, final state should reflect just F2 being added and F1 in deleted state, irrespective of whether compaction is complete or not. 

       

       

       

        Attachments

          Activity

            People

            • Assignee:
              shivnarayan sivabalan narayanan
              Reporter:
              shivnarayan sivabalan narayanan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: