Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7127 Fetch-on-demand metadata for the impalad-side catalog
  3. IMPALA-7509

Create table after drop can lead to table not found exception

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.1.0
    • Component/s: None
    • Labels:
      None
    • Epic Color:
      ghx-label-9

      Description

      There are two impalads. One running with old mode and one with fetch from catalogd mode.

      Now create a table and drop it in the impalad running in old mode. Following this create the same table from the new mode coodinator. It sometimes throws table not found exception.

      Test Create and Drop of a table in V1
        Running  1  iterations
      Going for :  CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
      Going for :  DROP TABLE PARTS;
      Test Create and Drop of a table in V2
        Running  1  iterations
      Going for :  CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
      Traceback (most recent call last):
        File "testing.py", line 21, in execute_query
          cursor.execute(query)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
          configuration=configuration)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
          self._execute_async(op)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
          operation_fn()
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
          async=True)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
          return self._operation('ExecuteStatement', req)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
          resp = self._rpc(kind, request)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
          err_if_rpc_not_ok(response)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
          raise HiveServer2Error(resp.status.errorMessage)
      HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
      CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))

      Going for :  DROP TABLE PARTS;
      Traceback (most recent call last):
        File "testing.py", line 21, in execute_query
          cursor.execute(query)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
          configuration=configuration)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
          self._execute_async(op)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
          operation_fn()
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
          async=True)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
          return self._operation('ExecuteStatement', req)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
          resp = self._rpc(kind, request)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
          err_if_rpc_not_ok(response)
        File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
          raise HiveServer2Error(resp.status.errorMessage)
      HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
      CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))

      Corresponding Impalad log will attached as well.

        Attachments

        1. failure_snippet.txt
          24 kB
          Rituparna Agrawal

          Activity

            People

            • Assignee:
              vukercegovac Vuk Ercegovac
              Reporter:
              parna Rituparna Agrawal
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: