[FLINK-5750] Incorrect translation of n-ary Union

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.0, 1.3.4, 1.4.2, 1.5.0, 1.6.0
• Fix Version/s: 1.4.3, 1.5.3, 1.6.0, 1.7.0
• Component/s: Table SQL / API
• Labels: pull-request-available
• Assignee: Alexander Koltsov
• Reporter: Anton Mushin
• Created: 09/Feb/17 09:09
• Updated: 24/Jul/18 13:06
• Resolved: 24/Jul/18 13:06

Description

Calcite's union operator is supports more than two input relations. However, Flink's translation rules only consider the first two relations because we assumed that Calcite's union is binary.
This problem exists for batch and streaming queries.

It seems that Calcite only generates non-binary Unions in rare cases ((SELECT * FROM t) UNION ALL (SELECT * FROM t) UNION ALL (SELECT * FROM t) results in two binary union operators) but the problem definitely needs to be fixed.

The following query can be used to validate the problem.

@Test
	public void testValuesWithCast() throws Exception {
		ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
		BatchTableEnvironment tableEnv = TableEnvironment.getTableEnvironment(env, config());

		String sqlQuery = "VALUES (1, cast(1 as BIGINT) )," +
			"(2, cast(2 as BIGINT))," +
			"(3, cast(3 as BIGINT))";
		String sqlQuery2 = "VALUES (1,1)," +
			"(2, 2)," +
			"(3, 3)";
		Table result = tableEnv.sql(sqlQuery);
		DataSet<Row> resultSet = tableEnv.toDataSet(result, Row.class);
		List<Row> results = resultSet.collect();

		Table result2 = tableEnv.sql(sqlQuery2);
		DataSet<Row> resultSet2 = tableEnv.toDataSet(result2, Row.class);
		List<Row> results2 = resultSet2.collect();

		String expected = "1,1\n2,2\n3,3";
		compareResultAsText(results2, expected);
		compareResultAsText(results, expected);
	}

AR for results variable

java.lang.AssertionError: Different elements in arrays: expected 3 elements and received 2
 expected: [1,1, 2,2, 3,3]
 received: [1,1, 2,2] 
Expected :3
Actual   :2

Issue Links

• links to: GitHub Pull Request #6267
• links to: GitHub Pull Request #6287
• links to: GitHub Pull Request #6341