[IMPALA-11845] Select STAR with table ref is not resolved correctly on column/row masked views

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0, Impala 3.4.0, Impala 3.4.1, Impala 4.1.0, Impala 4.2.0, Impala 4.1.1
• Fix Version/s: Impala 4.1.2, Impala 4.3.0
• Component/s: Security
• Target Version: Impala 4.1.2
• Epic Color: ghx-label-12
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 17/Jan/23 02:57
• Updated: 28/Mar/23 22:02
• Resolved: 01/Feb/23 09:32

Description

When a catalog view is applied with Ranger column-masking or row-filtering policies, select STAR on it doesn't work if the table ref is used, e.g. this doesn't work:

select v.* from masked_view v;

ERROR: IllegalStateException: null

Note that selecting STAR directly or selecting any columns still work:

select * from masked_view;
select c1, c2 from masked_view;

The stacktrace:

I0117 10:50:36.894714 21116 jni-util.cc:288] f940f87cdea03526:3c48d0f300000000] java.lang.IllegalStateException 
        at com.google.common.base.Preconditions.checkState(Preconditions.java:486)
        at org.apache.impala.analysis.Analyzer.resolvePathWithMasking(Analyzer.java:1216)
        at org.apache.impala.analysis.Analyzer.resolvePathWithMasking(Analyzer.java:1177)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyzeStarPath(SelectStmt.java:785)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.collectStarExpandedPaths(SelectStmt.java:879)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:333)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:280)
        at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:272)
        at org.apache.impala.analysis.AnalysisContext.reAnalyze(AnalysisContext.java:622)
        at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:553)
        at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2059)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1967)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1789)
        at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)

Related code:

1180   public Path resolvePathWithMasking(List<String> rawPath, PathType pathType,
1181       TimeTravelSpec timeTravelSpec) throws AnalysisException, TableLoadingException {
1182     Path resolvedPath = resolvePath(rawPath, pathType, timeTravelSpec);
1183     // Skip normal resolution cases that don't relate to nested types.
1184     if (pathType == PathType.TABLE_REF) {
1185       if (resolvedPath.destTable() != null || !resolvedPath.isRootedAtTuple()) {
1186         return resolvedPath;
1187       }
1188     } else if (pathType == PathType.SLOT_REF) {
1189       if (!resolvedPath.getMatchedTypes().get(0).isStructType()) {
1190         return resolvedPath;
1191       }
1192     } else if (pathType == PathType.STAR) {
1193       if (!resolvedPath.destType().isStructType() || !resolvedPath.isRootedAtTuple()) {
1194         return resolvedPath;
1195       }
1196     }
1197     // In this case, resolvedPath is resolved on a nested column. Check if it's resolved
1198     // on a table masking view. The root TableRef(table/view) could be at a parent query
1199     // block (correlated case, e.g. "t.int_array" in query
1200     // "SELECT ... FROM tbl t, (SELECT * FROM t.int_array) a" roots at "tbl t" which is
1201     // in the parent block), so we should find the parent block first then we can find
1202     // the root TableRef.
1203     TupleId rootTupleId = resolvedPath.getRootDesc().getId();
1204     Analyzer parentAnalyzer = findAnalyzer(rootTupleId);
1205     TableRef rootTblRef = parentAnalyzer.getTableRef(rootTupleId);
1206     Preconditions.checkNotNull(rootTblRef);
1207     if (!rootTblRef.isTableMaskingView()) return resolvedPath;
1208     // resolvedPath is resolved on a nested column of a table masking view. The view
1209     // won't produce results of nested columns. It just exposes the nested columns of the
1210     // underlying BaseTableRef in the fields of its output type. (See more in
1211     // InlineViewRef#createTupleDescriptor()). We need to resolve 'rawPath' inside the
1212     // view as if the underlying table is not masked. So the resolved path can point to
1213     // the real table and be used to create materialized slot in the TupleDescriptor of
1214     // the real table.
1215     InlineViewRef tableMaskingView = (InlineViewRef) rootTblRef;
1216     Preconditions.checkState(
1217         tableMaskingView.getUnMaskedTableRef() instanceof BaseTableRef);
1218     // Resolve rawPath inside the table masking view to point to the real table.
1219     Path maskedPath = tableMaskingView.inlineViewAnalyzer_.resolvePath(
1220         rawPath, pathType, timeTravelSpec);
1221     maskedPath.setPathBeforeMasking(resolvedPath);
1222     return maskedPath;
1223   }

https://github.com/apache/impala/blob/b0009db40b7a532694987a0f4280b322d72f07b7/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L1217

The 'pathType' is STAR and the 'rawPath' is the view ref. It's not a nested struct column so should be handled in the if-branches at the begining.

Issue Links

• is caused by: IMPALA-9330 Support masking queries containing correlated collection references
• links to: Code Review