[IMPALA-11281] Consider loading the table metadata for a ResetMetadataStmt - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: Impala 4.2.0, Impala 4.1.1
Component/s: None
Labels:
None

Epic Color:
ghx-label-10

Description

Currently when a ResetMetadataStmt that has a non-null 'tableName_' is being analyzed, we do not add its 'tableName_' to the given 'tblRefs' in collectTableRefs() if its 'partitionSpec_' is null (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131).

When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding AuthorizableTable (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229) since the table is an IncompleteTable.

If the column names are not populated in the corresponding AuthorizableTable of a table in a ResetMetadataStmt, then the logic in RangerAuthorizationChecker#authorizeByTableMasking() that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case ((AuthorizableTable) authorizable).getColumns() returns an empty list. As a result, such an update would be allowed if there is no other authorization error.

To reproduce the issue, we could comment out all the test cases in RangerAuditLogTest#testAuditsForColumnMasking() except for the following one. The following test case would fail since the query "invalidate metadata functional.alltypestiny" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to SelectStmt#collectTableRefs() that triggers the metadata loading of the table 'functional.alltypestiny'.

      // Updates on metadata fails by column-masking policies.
      authzError(events -> {
        assertEquals(1, events.size());
        assertEquals("invalidate metadata functional.alltypestiny",
            events.get(0).getRequestData());
        assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
            events.get(0));
        // Make sure it's denied by a column masking policy.
        assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
      }, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));