[HIVE-19870] HCatalog dynamic partition query can fail, if the table path is managed by Sentry

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Assignee: Peter Vary
• Reporter: Peter Vary
• Created: 12/Jun/18 16:10
• Updated: 17/Nov/22 08:55
• Resolved: 19/Jun/18 08:00

Description

The central issue is that HCatalog is assuming it needs to handle the storage based authorization features. When a job completes, in HCatalog's file committing phase it tries to manually set the permissions for the table for authorization's sake, which makes it go against auto-authorization managment features provided by Sentry.

The offending code is specifically at https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L373-L374

and

https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L380-L385

and

https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L952-L954

where you can notice that after it moves the files created by the job into their respective partition directories under the final table destination, it goes onto trying to perform chmod/chgrp operations which will fail out