Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-654

Calls to append_partition fail when Sentry is enabled

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      CDH 5.3.1

      Description

      When the Hive Metastore has org.apache.sentry.binding.metastore.MetastoreAuthzBinding configured as a pre event listener class, calls to the metastore Thrift API's append_partition method fail with a MetaException whose message is "java.lang.NullPointerException". In practice, this causes LOAD DATA INPATH statements targeting particular partitions of a partitioned table to fail. Example stack trace:

      Exception in thread "main" MetaException(message:java.lang.NullPointerException)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
      at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
      at Test.main(Test.java:31)

      The likely cause of this is that HiveMetastoreAuthzBinding's authorizeAddPartition method, which is triggered on both add_partition calls and append_partition calls, assumes that the added Partition object's storage descriptor is not null. However, org.apache.hadoop.hive.metastore.HiveMetaStore's append_partition_common method creates a Partition object on the fly based on the provided database, table, and partition values, and it doesn't initialize that object's storage descriptor until after it has already called firePreEvent (thereby signaling the pre event listeners to execute).

        Attachments

        1. SENTRY-654.patch
          2 kB
          Ryan P
        2. SENTRY-654.patch
          8 kB
          Ryan P

          Issue Links

            Activity

              People

              • Assignee:
                Ryan P Ryan P
                Reporter:
                Richard Williams Richard Williams
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: