[SENTRY-703] Calls to add_partition fail when passed a Partition object with a null location

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.6.0

Environment

CDH 5.3.3

• Assignee: Prasad Suresh Mujumdar
• Reporter: Richard Williams
• Created: 20/Apr/15 22:27
• Updated: 22/Apr/15 16:16
• Resolved: 22/Apr/15 16:12

Description

When the Hive Metastore has org.apache.sentry.binding.metastore.MetastoreAuthzBinding configured as a pre event listener class, calls to the metastore Thrift API's add_partition and add_partitions methods fail with a MetaException whose message is "java.lang.NullPointerException" if they are passed Partition objects whose StorageDescriptors have null location fields. In practice, this causes "alter table add partition" statements that do not specify a location and are issued through the Hive CLI (though apparently not HiveServer2) to fail. Example stack trace:

Exception in thread "main" MetaException(message:java.lang.NullPointerException)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result$add_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:36240)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result$add_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:36199)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result.read(ThriftHiveMetastore.java:36122)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_add_partition_with_environment_context(ThriftHiveMetastore.java:1222)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_partition_with_environment_context(ThriftHiveMetastore.java:1208)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.add_partition(HiveMetaStoreClient.java:439)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.add_partition(HiveMetaStoreClient.java:433)
        at hivetest.Main.main(Main.java:190)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

The likely cause of this is SENTRY-654. The patch that was committed for that issue changed HiveMetastoreAuthzBinding's authorizeAddPartition to check whether or not the StorageDescriptor nested inside the passed Partition object was null before calling its getLocation method, thereby preventing append_partition calls from failing with a NullPointerException. However, in doing so, it removed a check that was ensuring that the location itself was not null (!StringUtils.isEmpty(partitionLocation)). It appears that both checks are necessary.

Issue Links

• is related to: SENTRY-654 Calls to append_partition fail when Sentry is enabled