Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3917

Add max segment merge span to cleanup intermediate data of cube building

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently the intermediate data of building cube can not be deleted for it maybe used for later cubing merging. But it result in double space used in HDFS.

      In actual scenario, we only need month-level segment span in maximum.

      So if a span of segment is larger than a month, we think it don't need be merged and the intermediate data can be deleted.

      So we can add a config kylin.cube.max-segment-merge.span,  default is -1.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liushaohui Shaohui Liu
                Reporter:
                liushaohui Shaohui Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: