Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-886

Data Retention for streaming data

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None
    • Sprint:
      Sprint 47, Sprint 48

      Description

      Streaming data will grow very fast and consume huge storage. since queries are focus on recent data only in most of cases, support data retention in streaming data model is a critical feature to control storage size and even query performance.

      1. Add one config setting option in streaming data model
      2. User could define how long to keep data, for example, 7 days, 30 days.
      3. This is management option which could be updated on the fly
      4. Default retention option should be controlled by kylin.properteis file.

        Attachments

          Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. KYLIN-895 Add "retention_range" attribute for cube instance, and automatically drop the oldest segment when exceeds retention

          • Major - Major loss of function.
          • Closed
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. KYLIN-906 cube retention

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                shaofengshi Shao Feng Shi
                Reporter:
                lukehan Luke Han
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: