[IMPALA-9010] Support pre-defined mask types from Ranger UI

• Type: New Feature
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: Impala 3.4.0
• Component/s: Frontend
• Epic Link: Ranger Column Masking
• Epic Color: ghx-label-13
• Assignee: Quanlong Huang
• Reporter: Kurt Deschler
• Created: 04/Oct/19 23:36
• Updated: 10/Dec/20 07:49
• Resolved: 17/Jan/20 22:48

Description

Review Hive implementation/behavior.

Redact/Partial/Hash/Nullify/Unmasked/Date
These will be implemented as static SQL transforms in Impala

To be specifit, we need to implement 6 builtin functions:

• mask: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMask.java
• mask_hash: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
• mask_first_n: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskFirstN.java
• mask_last_n: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskLastN.java
• mask_show_first_n: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskShowFirstN.java
• mask_show_last_n: https://github.com/apache/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskShowLastN.java

These are Hive GenericUDFs which Impala can't use. So we have to create our own builtin functions.

Issue Links

• is related to: IMPALA-10387 Implement missing overloads of mask functions used in Ranger default masking policies
• links to: Code Review