[IMPALA-11281] Consider loading the table metadata for a ResetMetadataStmt

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Epic Color: ghx-label-10
• Assignee: Fang-Yu Rao
• Reporter: Fang-Yu Rao
• Created: 04/May/22 21:59
• Updated: 29/Sep/23 06:41
• Resolved: 25/Aug/22 01:32

Description

Currently when a ResetMetadataStmt that has a non-null 'tableName_' is being analyzed, we do not add its 'tableName_' to the given 'tblRefs' in collectTableRefs() if its 'partitionSpec_' is null (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131).

When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding AuthorizableTable (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229) since the table is an IncompleteTable.

If the column names are not populated in the corresponding AuthorizableTable of a table in a ResetMetadataStmt, then the logic in RangerAuthorizationChecker#authorizeByTableMasking() that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case ((AuthorizableTable) authorizable).getColumns() returns an empty list. As a result, such an update would be allowed if there is no other authorization error.

To reproduce the issue, we could comment out all the test cases in RangerAuditLogTest#testAuditsForColumnMasking() except for the following one. The following test case would fail since the query "invalidate metadata functional.alltypestiny" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to SelectStmt#collectTableRefs() that triggers the metadata loading of the table 'functional.alltypestiny'.

      // Updates on metadata fails by column-masking policies.
      authzError(events -> {
        assertEquals(1, events.size());
        assertEquals("invalidate metadata functional.alltypestiny",
            events.get(0).getRequestData());
        assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
            events.get(0));
        // Make sure it's denied by a column masking policy.
        assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
      }, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));

Issue Links

• is related to: IMPALA-11501 Add flag to allow metadata-cache operations on masked tables
• relates to: IMPALA-10554 Block modifications when row-filter/column-mask is enabled for the user