Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-703

Calls to add_partition fail when passed a Partition object with a null location

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      CDH 5.3.3

      Description

      When the Hive Metastore has org.apache.sentry.binding.metastore.MetastoreAuthzBinding configured as a pre event listener class, calls to the metastore Thrift API's add_partition and add_partitions methods fail with a MetaException whose message is "java.lang.NullPointerException" if they are passed Partition objects whose StorageDescriptors have null location fields. In practice, this causes "alter table add partition" statements that do not specify a location and are issued through the Hive CLI (though apparently not HiveServer2) to fail. Example stack trace:

      Exception in thread "main" MetaException(message:java.lang.NullPointerException)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result$add_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:36240)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result$add_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:36199)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partition_with_environment_context_result.read(ThriftHiveMetastore.java:36122)
              at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_add_partition_with_environment_context(ThriftHiveMetastore.java:1222)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_partition_with_environment_context(ThriftHiveMetastore.java:1208)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.add_partition(HiveMetaStoreClient.java:439)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.add_partition(HiveMetaStoreClient.java:433)
              at hivetest.Main.main(Main.java:190)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:497)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      
      

      The likely cause of this is SENTRY-654. The patch that was committed for that issue changed HiveMetastoreAuthzBinding's authorizeAddPartition to check whether or not the StorageDescriptor nested inside the passed Partition object was null before calling its getLocation method, thereby preventing append_partition calls from failing with a NullPointerException. However, in doing so, it removed a check that was ensuring that the location itself was not null (!StringUtils.isEmpty(partitionLocation)). It appears that both checks are necessary.

        Attachments

        1. SENTRY-703.1.patch
          3 kB
          Prasad Suresh Mujumdar

          Issue Links

            Activity

              People

              • Assignee:
                prasadm Prasad Suresh Mujumdar
                Reporter:
                Richard Williams Richard Williams
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: