Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
Description
We can introduce richer merge strategies, one of which is already introduced is PartialUpdateMergeFunction, which completes non-NULL fields when merging. We can introduce more powerful merge strategies, such as support for pre-aggregated merges.
Usage 1:
CREATE TABLE T (
pk STRING PRIMARY KEY NOT ENFOCED,
sum_field1 BIGINT,
sum_field1 BIGINT
) WITH (
'merge-engine' = 'aggregation',
'sum_field1.aggregate-function' = 'sum',
'sum_field2.aggregate-function' = 'sum'
);
INSERT INTO T VALUES ('pk1', 1, 1);
INSERT INTO T VALUES ('pk1', 1, 1);
SELECT * FROM T;
=> output 'pk1', 2, 2
Usage 2:
CREATE MATERIALIZED VIEW T
with (
'merge-engine' = 'aggregation'
) AS SELECT
pk,
SUM(field1) AS sum_field1,
SUM(field2) AS sum_field1
FROM source_t
GROUP BY pk ;
This will start a stream job to synchronize data, consume source data, and write incrementally to T. This data synchronization job has no state.
Attachments
Issue Links
- relates to
-
FLINK-29189 Pass KeyValue to MergeFunction
-
- Closed
-
-
-
- Closed
-
- links to