Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5750

Incorrect translation of n-ary Union

    XMLWordPrintableJSON

    Details

      Description

      Calcite's union operator is supports more than two input relations. However, Flink's translation rules only consider the first two relations because we assumed that Calcite's union is binary.
      This problem exists for batch and streaming queries.

      It seems that Calcite only generates non-binary Unions in rare cases ((SELECT * FROM t) UNION ALL (SELECT * FROM t) UNION ALL (SELECT * FROM t) results in two binary union operators) but the problem definitely needs to be fixed.

      The following query can be used to validate the problem.

      @Test
      	public void testValuesWithCast() throws Exception {
      		ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      		BatchTableEnvironment tableEnv = TableEnvironment.getTableEnvironment(env, config());
      
      		String sqlQuery = "VALUES (1, cast(1 as BIGINT) )," +
      			"(2, cast(2 as BIGINT))," +
      			"(3, cast(3 as BIGINT))";
      		String sqlQuery2 = "VALUES (1,1)," +
      			"(2, 2)," +
      			"(3, 3)";
      		Table result = tableEnv.sql(sqlQuery);
      		DataSet<Row> resultSet = tableEnv.toDataSet(result, Row.class);
      		List<Row> results = resultSet.collect();
      
      		Table result2 = tableEnv.sql(sqlQuery2);
      		DataSet<Row> resultSet2 = tableEnv.toDataSet(result2, Row.class);
      		List<Row> results2 = resultSet2.collect();
      
      		String expected = "1,1\n2,2\n3,3";
      		compareResultAsText(results2, expected);
      		compareResultAsText(results, expected);
      	}
      

      AR for results variable

      java.lang.AssertionError: Different elements in arrays: expected 3 elements and received 2
       expected: [1,1, 2,2, 3,3]
       received: [1,1, 2,2] 
      Expected :3
      Actual   :2
      

        Attachments

          Activity

            People

            • Assignee:
              Alexander_Koltsov Alexander Koltsov
              Reporter:
              anmu Anton Mushin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: