Uploaded image for project: 'Apache Apex Malhar'
  1. Apache Apex Malhar
  2. APEXMALHAR-2271

Special SpillableSetMultimap<K, Window.SessionWindow<K>> handling in ManagedState in SpillableSessionWindowedStorage

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      The problem with this data structure is that the key is not time based.
      This poses a problem because we will have a problem purging.
      One idea is to have `put` write the key and the window set to the timebucket that the latest window in that set belongs to, which means possible duplication of the key and the window set, and make `get` do a for loop on all time buckets to find the key. That way, purging based on the time bucket can still be done.

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. APEXMALHAR-2130 Scalable windowed storage

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue APEXMALHAR-2244 Optimize WindowedStorage and Spillable data structures for time series

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #495

          Web Link GitHub Pull Request #495

          Activity

            People

              hsy541 Siyuan Hua
              davidyan David Yan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: