Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
we can finish the segment load after receives configurable amount of data and create a new segment to load new streaming data. While finishing the segment we can trigger the agg tables on it. So there would not be any agg tables on ongoing streaming segment but querying can be done on the streaming segment of actual table.
For example user configures stream_segment size as 1 GB, so for every 1 GB of stream data it receives it creates a new segment and finishes the current segment. While finishing the current segment we can trigger agg table loading and compaction of segments.
While querying of data we change the query plan to apply union of agg table and streaming segment of actual table to get the current data.
Attachments
Issue Links
- links to
