[IMPALA-9148] AuthorizationStmtTest.testColumnMaskEnabled seems flaky - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Duplicate
Affects Version/s: None
Fix Version/s: None
Component/s: Frontend
Labels:
None

Target Version:

Impala 3.4.0
Epic Color:
ghx-label-7

Description

Recently we have seen failed AuthorizationStmtTest.testColumnMaskEnabled occasionally with the following error messages (e.g., https://jenkins.impala.io/job/gerrit-verify-dryrun/5194/consoleFull).

Impala does not support row filtering yet. Row filtering is enabled on table: functional.alltypes_view
expected:
Impala does not support column masking yet. Column masking is enabled on column: functional.alltypes_view.string_col

Taking a look at the testColumnMaskEnabled(), we can see the related SQL statement is

select string_col from functional.alltypes_view;

I found that for this SQL statement, authorizeRowFilterAndColumnMask() in RangerAuthorizationCheker.java will be called first (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L183-L200). There will be two privilege requests, one request for column, and the other for table. The function authorizeRowFilter() is the only function that could produce the error message above (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L295-L308). Specifically, this error would be generated if plugin_.evalRowFilterPolicies(req, null).isRowFilterEnabled() returns true (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L303).

I have taken a brief look at isRowFilterEnabled(), and found that it will return true only if there is some policy on the Ranger server that specifies the policy of row filtering (according to my current understanding). However, in testColumnMaskEnabled() (https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java#L2836), we only add a policy for column masking. Therefore, I suspect it may be possible that some other tests added to the Ranger server some policy for row filtering but did not properly do the cleanup of this row filtering policy afterwards.

To address this issue, we should add some logic to clean up the policies stored on the Ranger server before running this JUnit test. This JUnit test assumes that the Ranger server does not store any policies related to column masking and row filtering before the testing.