Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21388

Constant UDF is not pushed to JDBCStorage Handler

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0.0
    • Fix Version/s: 3.2.0, 4.0.0
    • Component/s: CBO, StorageHandler
    • Labels:
      None

      Description

      A query involve a Hive UDF which produce a constant value does not push to JDBC table. Replacing the UDF with a constant push down works. Ideally, Hive shall first do constant folding and then push the computation.

      Here is the example:

      explain select PRINCIPAL_NAME from sys.TBL_PRIVS where PRINCIPAL_NAME=current_user();
      +----------------------------------------------------+
      |                      Explain                       |
      +----------------------------------------------------+
      | Plan optimized by CBO.                             |
      |                                                    |
      | Stage-0                                            |
      |   Fetch Operator                                   |
      |     limit:-1                                       |
      |     Select Operator [SEL_3]                        |
      |       Output:["_col0"]                             |
      |       Filter Operator [FIL_2]                      |
      |         predicate:(_col5 = 'hrt_qa')               |
      |         Select Operator [SEL_1]                    |
      |           Output:["_col5"]                         |
      |           TableScan [TS_0]                         |
      |             Output:["principal_name"],properties:{"hive.sql.query":"SELECT `tbl_grant_id`, `create_time`, `grant_option`, `grantor`, `grantor_type`, `principal_name`, `principal_type`, `tbl_priv`, `tbl_id`, `authorizer`\nFROM `TBL_PRIVS`","hive.sql.query.fieldNames":"tbl_grant_id,create_time,grant_option,grantor,grantor_type,principal_name,principal_type,tbl_priv,tbl_id,authorizer","hive.sql.query.fieldTypes":"bigint,int,int,string,string,string,string,string,bigint,string","hive.sql.query.split":"true"} |
      |                                                    |
      +----------------------------------------------------+
      

      If I replace current_user() with a constant, the predicate is pushed to table scan.

      Also, setting annotation deterministic=true and make initialize() return a ConstantObjectInspector of GenericUDFCurrentUser does not make a difference.

        Attachments

        1. HIVE-21388.01.patch
          11 kB
          Jesus Camacho Rodriguez
        2. HIVE-21388.02.patch
          11 kB
          Jesus Camacho Rodriguez
        3. HIVE-21388.patch
          9 kB
          Jesus Camacho Rodriguez

          Activity

            People

            • Assignee:
              jcamachorodriguez Jesus Camacho Rodriguez
              Reporter:
              daijy Daniel Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: