Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3424

Use of "DDL delegates" can lead to unexpected behavior (managed tables not being dropped)

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Not A Bug
    • Kudu_Impala
    • Product Backlog
    • Catalog

    Description

      A "ddl delegate" provides ddl related functionality that isn't in Hive. For example Kudu tables use ddl delegate to create and drop tables in Kudu. If a user were to somehow create a Kudu table in Hive and that table metadata wasn't loaded in Impala, then the user dropped the entire database using "cascade" through Impala, then the Kudu table would remain. Eventually Hive should support Kudu tables and the "ddl delegate" can be removed.

      Attachments

        Activity

          mjacobs Matthew Jacobs added a comment -

          Marking as critical because we can hit this pretty easily.

          mjacobs Matthew Jacobs added a comment - Marking as critical because we can hit this pretty easily.
          mjacobs Matthew Jacobs added a comment -

          This JIRA is pretty ambiguous.

          Here's at least one case we should handle:

          Previously created managed kudu table 'managed_kudu' and then removed it. Then it cannot be used or dropped by Impala:

          [localhost:21000] > select * from managed_kudu;
          Query: select * from managed_kudu
          Query submitted at: 2016-09-21 11:41:32 (Coordinator: http://mj-desktop.ca.cloudera.com:25000)
          ERROR: ImpalaRuntimeException: Unable to initialize the Kudu scan node
          CAUSED BY: NonRecoverableException: The table does not exist: table_name: "managed_kudu"
          
          [localhost:21000] > drop table managed_kudu;
          Query: drop table managed_kudu
          ERROR: 
          TableNotFoundException: Table 'managed_kudu' does not exist in Kudu master(s) '0.0.0.0:7051'.
          
          
          mjacobs Matthew Jacobs added a comment - This JIRA is pretty ambiguous. Here's at least one case we should handle: Previously created managed kudu table 'managed_kudu' and then removed it. Then it cannot be used or dropped by Impala: [localhost:21000] > select * from managed_kudu; Query: select * from managed_kudu Query submitted at: 2016-09-21 11:41:32 (Coordinator: http: //mj-desktop.ca.cloudera.com:25000) ERROR: ImpalaRuntimeException: Unable to initialize the Kudu scan node CAUSED BY: NonRecoverableException: The table does not exist: table_name: "managed_kudu" [localhost:21000] > drop table managed_kudu; Query: drop table managed_kudu ERROR: TableNotFoundException: Table 'managed_kudu' does not exist in Kudu master(s) '0.0.0.0:7051' .
          mjacobs Matthew Jacobs added a comment -

          I think we can close this as not an issue. The case that I mentioned above should be handled by changing the table to be an external table and then dropping it. I'm not aware of any specific problems that we want to make code changes for. dTsirogiannis do you agree?

          mjacobs Matthew Jacobs added a comment - I think we can close this as not an issue. The case that I mentioned above should be handled by changing the table to be an external table and then dropping it. I'm not aware of any specific problems that we want to make code changes for. dTsirogiannis do you agree?

          mjacobs, yes. This case should be handled correctly now.

          dtsirogiannis Dimitris Tsirogiannis added a comment - mjacobs , yes. This case should be handled correctly now.
          mjacobs Matthew Jacobs added a comment -

          We're not aware of any cases we're not handling here.

          mjacobs Matthew Jacobs added a comment - We're not aware of any cases we're not handling here.

          People

            dtsirogiannis Dimitris Tsirogiannis
            caseyc casey
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: