Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9544

Error dropping fully qualified partitioned table - Internal error processing get_partition_names

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.14.0
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      HDP 2.2

      Description

      When attempting to drop a partitioned table using a fully qualified name I get this error:

      hive -e 'drop table myDB.my_table_name;'
      Logging initialized using configuration in file:/etc/hive/conf/hive-log4j.properties
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hive/lib/hive-jdbc-0.14.0.2.2.0.0-2041-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
      
      
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.thrift.TApplicationException: Internal error processing get_partition_names
      

      It succeeds if I instead do:

      hive -e 'use myDB; drop table my_table_name;'

      Regards,

      Hari Sekhon
      http://www.linkedin.com/in/harisekhon

        Activity

        Hide
        ctang.ma Chaoyu Tang added a comment -

        It is possible since I was not able to reproduce this issue in current upstream.

        Show
        ctang.ma Chaoyu Tang added a comment - It is possible since I was not able to reproduce this issue in current upstream.
        Hide
        Ryan P Ryan P added a comment -

        This looks pretty similar to HIVE-10421. Are the offending tables partitioned?

        Show
        Ryan P Ryan P added a comment - This looks pretty similar to HIVE-10421 . Are the offending tables partitioned?
        Hide
        ErwanMAS ErwanMAS added a comment -

        I have this bug , on Hortonworks 2.2.4 , so with hive 0.14

        Show
        ErwanMAS ErwanMAS added a comment - I have this bug , on Hortonworks 2.2.4 , so with hive 0.14
        Hide
        Ryan P Ryan P added a comment -

        Hari Sekhon that's fair, sorry I missed that some how. I don't think the issue is with Sentry it's so much as the presence of a pre-event listener.

        From the HMS logs I was able to find the following

        ERROR org.apache.thrift.ProcessFunction: Internal error processing get_partition_names
        NoSuchObjectException(message:default.testdrop table not found)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.fireReadTablePreEvent(HiveMetaStore.java:2820)

        Tracing the stack in the HMS logs I found that the error is thrown here:

        private void fireReadTablePreEvent(String dbName, String tblName) throws MetaException, NoSuchObjectException {
        if(preListeners.size() > 0) {
        // do this only if there is a pre event listener registered (avoid unnecessary
        // metastore api call)
        Table t = getMS().getTable(dbName, tblName);
        if (t == null) {
        throw new NoSuchObjectException(dbName + "." + tblName
        + " table not found");
        }
        firePreEvent(new PreReadTableEvent(t, this));
        }
        }

        I'm not sure why or how the database qualifier is lost but in the absence of a pre-event listener everything works as expected.

        Show
        Ryan P Ryan P added a comment - Hari Sekhon that's fair, sorry I missed that some how. I don't think the issue is with Sentry it's so much as the presence of a pre-event listener. From the HMS logs I was able to find the following ERROR org.apache.thrift.ProcessFunction: Internal error processing get_partition_names NoSuchObjectException(message:default.testdrop table not found) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.fireReadTablePreEvent(HiveMetaStore.java:2820) Tracing the stack in the HMS logs I found that the error is thrown here: private void fireReadTablePreEvent(String dbName, String tblName) throws MetaException, NoSuchObjectException { if(preListeners.size() > 0) { // do this only if there is a pre event listener registered (avoid unnecessary // metastore api call) Table t = getMS().getTable(dbName, tblName); if (t == null) { throw new NoSuchObjectException(dbName + "." + tblName + " table not found"); } firePreEvent(new PreReadTableEvent(t, this)); } } I'm not sure why or how the database qualifier is lost but in the absence of a pre-event listener everything works as expected.
        Hide
        harisekhon Hari Sekhon added a comment -

        Ryan Pridgeon No it was on Hortonworks so no Sentry.

        Show
        harisekhon Hari Sekhon added a comment - Ryan Pridgeon No it was on Hortonworks so no Sentry.
        Hide
        Ryan P Ryan P added a comment -

        Hari Sekhon were you using Sentry at the time? I personally could not reproduce this issue with Sentry disabled.

        Show
        Ryan P Ryan P added a comment - Hari Sekhon were you using Sentry at the time? I personally could not reproduce this issue with Sentry disabled.
        Hide
        dghosal Dipankar added a comment -

        Alternate way of doing this is :
        hive -hiveconf schema=mydb -e 'drop table $

        {hiveconf:schema}

        .my_table_name'

        I.e .. pass the schema/database name as hive conf.

        Show
        dghosal Dipankar added a comment - Alternate way of doing this is : hive -hiveconf schema=mydb -e 'drop table $ {hiveconf:schema} .my_table_name' I.e .. pass the schema/database name as hive conf.

          People

          • Assignee:
            ctang.ma Chaoyu Tang
            Reporter:
            harisekhon Hari Sekhon
          • Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development