Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-886

Data Retention for streaming data

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None
    • Sprint:
      Sprint 47, Sprint 48

      Description

      Streaming data will grow very fast and consume huge storage. since queries are focus on recent data only in most of cases, support data retention in streaming data model is a critical feature to control storage size and even query performance.

      1. Add one config setting option in streaming data model
      2. User could define how long to keep data, for example, 7 days, 30 days.
      3. This is management option which could be updated on the fly
      4. Default retention option should be controlled by kylin.properteis file.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shaofengshi Shao Feng Shi
                Reporter:
                lukehan Luke Han
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: