[IMPALA-9329] Table masking view should not have nested columns in select list

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 3.4.0
• Component/s: Security
• Epic Link: Ranger Column Masking
• Epic Color: ghx-label-3
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 27/Jan/20 15:13
• Updated: 05/Feb/20 05:48
• Resolved: 05/Feb/20 05:47

Description

If a table contains nested columns and column masking policies on some primitive columns, we should not add the nested columns in the table masking view.

To be specific, let's say table complextypestbl (id bigint, int_arr array<int>) has column masking policy on id: "id => id * 100". We will translate the AST of the following query

select t.id from complextypestbl t;

to the AST of the following query

select t.id from (
  select cast(id * 100 as BIGINT) as id,
    cast(int_array as ArrayType) as int_array
  from complextypestbl) t;

This causes a ParseError since we can't use ArrayType explicitly.

ERROR: ParseException: Syntax error in line 1:
SELECT CAST(int_array AS org.apache.impala.catalog.Ar...
                         ^
Encountered: IDENTIFIER
Expected: ARRAY, BIGINT, BINARY, BOOLEAN, CHAR, DATE, DATETIME, DECIMAL, REAL, FLOAT, INTEGER, MAP, SMALLINT, STRING, STRUCT, TIMESTAMP, TINYINT, VARCHAR

CAUSED BY: Exception: Syntax error

On the other hand, we don't support nested types in select list. So we should remove the redundant column for 'int_array' in the table masking view.

Issue Links

• relates to: IMPALA-9330 Support masking queries containing correlated collection references