[IMPALA-7509] Create table after drop can lead to table not found exception

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 3.1.0
• Epic Color: ghx-label-9
• Assignee: Vuk Ercegovac
• Reporter: Rituparna Agrawal
• Created: 29/Aug/18 22:01
• Updated: 12/Dec/18 00:09
• Resolved: 12/Dec/18 00:09

Description

There are two impalads. One running with old mode and one with fetch from catalogd mode.

Now create a table and drop it in the impalad running in old mode. Following this create the same table from the new mode coodinator. It sometimes throws table not found exception.

Test Create and Drop of a table in V1
  Running  1  iterations
Going for :  CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
Going for :  DROP TABLE PARTS;
Test Create and Drop of a table in V2
  Running  1  iterations
Going for :  CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
Traceback (most recent call last):
  File "testing.py", line 21, in execute_query
    cursor.execute(query)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
    configuration=configuration)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
    self._execute_async(op)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
    operation_fn()
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
    async=True)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
    return self._operation('ExecuteStatement', req)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
    resp = self._rpc(kind, request)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
    err_if_rpc_not_ok(response)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
    raise HiveServer2Error(resp.status.errorMessage)
HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))

Going for :  DROP TABLE PARTS;
Traceback (most recent call last):
  File "testing.py", line 21, in execute_query
    cursor.execute(query)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
    configuration=configuration)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
    self._execute_async(op)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
    operation_fn()
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
    async=True)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
    return self._operation('ExecuteStatement', req)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
    resp = self._rpc(kind, request)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
    err_if_rpc_not_ok(response)
  File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
    raise HiveServer2Error(resp.status.errorMessage)
HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))

Corresponding Impalad log will attached as well.