Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1883

Consensus Problem when running the tool, MetadataCleanupJob

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.5.4
    • Component/s: None
    • Labels:
      None

      Description

      When do the cleanup, current strategy is as follows:
      1. firstly create an referenceSet
      2. then add items not belonging to the referenceSet to the toDeleteSet
      3. finally delete those items in the toDeleteSet

      Consensus issue will occur since we cannot make sure that all of the items in toDeleteSet are not referenced in case that referenceSet changes during the process.

      For example, before the cleanup, SEGMENT_A is deleted and leave a DICT_A created at the building step. Then the referenceSet will not include DICT_A. After creating the reference set, SEGMENT_B is starting to build. Since DICT_A still exists, it can still be referenced by SEGMENT_B. Then DICT_A will still be included in the toDeleteSet and will be deleted later. Finally SEGMENT_B only owns a reference with no data.

        Attachments

          Activity

            People

            • Assignee:
              yaho Zhong Yanghong
              Reporter:
              yaho Zhong Yanghong
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: