Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7177

DataSetAggregateWithNullValuesRule fails creating null literal for non-nullable type

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.1
    • Fix Version/s: 1.4.0, 1.3.2
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      For example:

      flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/batch/sql/AggregationsITCase.scala
      @Test
        def testTableAggregationWithMultipleTableAPI(): Unit = {
          val env = ExecutionEnvironment.getExecutionEnvironment
          val tEnv = TableEnvironment.getTableEnvironment(env, config)
      
          val inputTable = CollectionDataSets.getSmallNestedTupleDataSet(env).toTable(tEnv, 'a, 'b)
          tEnv.registerDataSet("MyTable", inputTable)
      
          val result = tEnv.scan("MyTable")
            .where('a.get("_1") > 0)
            .select('a.get("_1").avg, 'a.get("_2").sum, 'b.count)
      
          val expected = "2,6,3"
          val results = result.toDataSet[Row].collect()
          TestBaseUtils.compareResultAsText(results.asJava, expected)
        }
      

        Activity

        Hide
        twalthr Timo Walther added a comment -

        Fixed as part of FLINK-7137.

        Show
        twalthr Timo Walther added a comment - Fixed as part of FLINK-7137 .
        Hide
        fhueske Fabian Hueske added a comment - - edited

        I looked into this issue, the problem is caused by the DataSetAggregateWithNullValuesRule, which tries to create a null literal for a non-nullable data type (field a of type RelRecordType in the given example).
        The purpose of the rule is to compute correct results for non-grouped aggregates if the input table is empty. For that it unions the input table with another table that has a single record with null literals. However, creating a null literal for a non-nullable data type fails.

        Show
        fhueske Fabian Hueske added a comment - - edited I looked into this issue, the problem is caused by the DataSetAggregateWithNullValuesRule , which tries to create a null literal for a non-nullable data type (field a of type RelRecordType in the given example). The purpose of the rule is to compute correct results for non-grouped aggregates if the input table is empty. For that it unions the input table with another table that has a single record with null literals. However, creating a null literal for a non-nullable data type fails.

          People

          • Assignee:
            twalthr Timo Walther
            Reporter:
            walterddr Rong Rong
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development