[IMPALA-12079] NoneType object is not iterable due to uncaught exceptions

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Clients
• Target Version: Impala 4.1.3
• Epic Color: ghx-label-6
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 21/Apr/23 00:14
• Updated: 4 days ago 21:36
• Resolved: 24/Apr/23 00:35

Description

We saw impala-shell crashed with the following error:

Query: ALTER table xxx recover partitions
Unknown Exception : 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/cs/cloudera/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1055.31529794/bin/../lib/impala-shell/impala_shell.py", line 1131, in _execute_stmt
    query_str, self.set_query_options)
  File "/cs/cloudera/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1055.31529794/lib/impala-shell/lib/impala_client.py", line 1069, in execute_query
    handle, rpc_status = self._do_beeswax_rpc(lambda: self.imp_service.query(query))
TypeError: 'NoneType' object is not iterable 

After adding some logs in "_do_beeswax_rpc()", we found it's due to an uncaught exception:

<class 'socket.error'> [Errno 104] Connection reset by peer

Here is a code sniper (after adding the debug log) of impala_client.py:

  def _do_beeswax_rpc(self, rpc, suppress_error_on_cancel=True):
    ...
    except Exception as e:
      ...
      print(type(e), str(e))  # Added this debug log
      if suppress_error_on_cancel and self.is_query_cancelled:
        raise QueryCancelledByShellException()
      else:
        if "BeeswaxException" in str(e):
          raise RPCException("ERROR: %s" % e.message)
        if "QueryNotFoundException" in str(e):
          raise QueryStateException('Error: Stale query handle')
      # The exception doesn't match any cases above so the method returns None

Unfortunately, I can't reproduce this in house and we didn't get chance to dig into where the exception come from. We should fix impala-shell anyway to expose the exception instead of returning None and crashing itself.

Issue Links

• is related to: IMPALA-12216 Print timestamp for impala-shell errors