Details
-
Bug
-
Status: Open
-
Not a Priority
-
Resolution: Unresolved
-
1.12.2
-
None
Description
Filters can remove +I messages and let a new row start with +U; similarly, they can remove the +U after -U which should instead have been reported as -D. This makes the semantics described on https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/streaming/dynamic_tables.html#table-to-stream-conversion wrong which makes users believe they can rely on +U vs. +I and -U vs. -D.
Either this needs to be fixed, or a disclaimer should be added to the respective paragraphs that +U/+I and -U/-D should be seem as the same.
Example:
CREATE TEMPORARY VIEW test_cardinality AS SELECT * FROM ( VALUES ('Alice', 'alice@test.com', ARRAY [ 'alice@test.com' ], 'Test Ltd'), ('Alice', 'alice@test.com', ARRAY [ 'alice@test.com' ], 'Test Ltd'), ('Alice', 'alice@test2.com', ARRAY [ 'alice@test.com', 'alice@test2.com' ], 'Test Ltd')) AS t ( name, email, aliases, company );
SELECT name, LEAD(CAST(company AS VARCHAR), 0) AS company FROM test_cardinality WHERE CARDINALITY(aliases) >= 2 GROUP BY name;
vs.
SELECT name, LEAD(CAST(company AS VARCHAR), 0) AS company FROM test_cardinality GROUP BY name HAVING COUNT(DISTINCT email) >= 2;