Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-29717 Supports hive udaf such as sum/count by native implementation
  3. FLINK-31193

The option `table.exec.hive.native-agg-function.enabled` should work at job level when using it in SqlClient side

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.17.0
    • 1.20.0
    • Connectors / Hive
    • None

    Description

      Sink Flink 1.17, we have implemented the native hive aggregation function, users can enable this optimization by the option `table.exec.hive.native-agg-function.enabled` per-job. The option works well in Table API & 

      HiveServer2Endpoint, but in SQL client, this option can't work per-job, it works at the module level.
      On the SqlClient side, if we want to use the native hive aggregation function, we need to enable the option first and then load HiveModule, only by doing that we can load the hive native aggregation function. This behavior is inconsistent with the definition of the option itself, and also is inconsistent with the behavior of the Table API and HiveServerEndpoint2, which is a bug. we cannot align the behavior of the option at the moment due to code implementation reasons. If we want to align the behavior, we need to modify the `Module` interface, which is a Public interface.  In 1.17 we didn't have time to complete this work, so we need to fix this in 1.18 and make the parameter per-job effective on the SqlClient side as well.

      Attachments

        Activity

          People

            Unassigned Unassigned
            lsy dalongliu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: