[IMPALA-9661] Avoid introducing unused columns in table masking view

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Frontend
• Epic Link: Ranger Column Masking
• Epic Color: ghx-label-3
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 16/Apr/20 00:36
• Updated: 28/Mar/23 22:03
• Resolved: 22/Mar/21 23:28

Description

If a table has column masking policies, we replace its unanalyzed TableRef with an analyzed InlineViewRef (table masking view) in FromClause.analyze(). However, we can't detect which columns are actually used in the original query at this point. In fact, analyze() for SelectList, WhereClause, GroupByClause and other clauses containing SlotRefs happen after FromClause.analyze(). After the whole query block is analyzed, we can get the exact set of required columns. We should do table masking there to avoid introducing unused columns.

To be specifit, if table tbl(id int, name string, address string) has column masking policies for column name and address to mask them, the following query

select name from tbl where id > 10;

will be rewritten to

select name from (
  select id, mask(name) as name, mask(address) as address from tbl
) tbl where id > 10;

The rewritten query introduce the requirement for SELECT privilege on the address column which isn't required by the original query. We should either fix this or IMPALA-9223.

Issue Links

• causes: IMPALA-11744 Table mask view should preserve the original column order in Hive
• links to: Code Review