Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21419

Partition Pruning not happening when using Apache Ranger masking

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.2
    • Fix Version/s: 3.0.0
    • Labels:
      None
    • Environment:
    • Target Version/s:

      Description

      I have a partitioned table, which I have a Ranger masking policy on a non-partition column. When I am attempting to query the table that includes the column that has masking enabled, then partition pruning no longer occurs. 

      To reproduce:

      Create two partitioned tables. I used TPC-H tables as they are publicly available and will provide the schemas and queries I used. Insert into the second table from the first table. For example:

      insert overwrite table analyst1.test1 PARTITION (l_shipdate)
      select l_commitdate, l_receiptdate, l_shipdate
      from analyst1.lineitem_partitioned
      where l_shipdate = '1992-01-02';

      I have attached the explain plan when a masking rule on l_commitdate is enabled and when not enabled.

      I have done a bit of deep dive and see that the pruning expression is not being set when the masking rule is enabled. 

        Attachments

        1. hive-jira-schema-explain-plan.txt
          42 kB
          Mert Hocanin
        2. Operators-in-debugger-with-masking.png
          57 kB
          Mert Hocanin
        3. Operators-in-debugger-without-masking.png
          59 kB
          Mert Hocanin

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mert_hoc Mert Hocanin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: