Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-886

Data Retention for streaming data

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • v1.4.0
    • v1.4.0
    • None
    • None
    • Sprint 47, Sprint 48

    Description

      Streaming data will grow very fast and consume huge storage. since queries are focus on recent data only in most of cases, support data retention in streaming data model is a critical feature to control storage size and even query performance.

      1. Add one config setting option in streaming data model
      2. User could define how long to keep data, for example, 7 days, 30 days.
      3. This is management option which could be updated on the fly
      4. Default retention option should be controlled by kylin.properteis file.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. KYLIN-895 Add "retention_range" attribute for cube instance, and automatically drop the oldest segment when exceeds retention

          • Major - Major loss of function.
          • Closed
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. KYLIN-906 cube retention

          • Major - Major loss of function.
          • Closed

          Activity

            People

              shaofengshi Shao Feng Shi
              lukehan Luke Han
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: