[IMPALA-10541] impyla fails to fetchall() in python3

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Clients
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: Balazs Jeszenszky
• Created: 23/Feb/21 11:35
• Updated: 23/Feb/21 11:35

Description

select * queries fail instantly, with:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 536, in fetchall
    return list(self)
  File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 582, in __next__
    self._buffer = self._last_operation.fetch(self.description,
  File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 187, in description
    schema = self._last_operation.get_result_schema()
  File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 1295, in get_result_schema
    entry = column.typeDesc.types[0].primitiveEntry
AttributeError: 'NoneType' object has no attribute 'types'

Selecting specific columns hangs indefinitely. The stack trace is always:

 File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 535, in fetchall
    return list(self)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 583, in __next__
    convert_types=self.convert_types)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1242, in fetch
    resp = self._rpc('FetchResults', req)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 992, in _rpc
    response = self._execute(func_name, request)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1009, in _execute
    return func(request)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 219, in _req
    return self._recv(_api)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 238, in _recv
    result.read(self._iprot)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 160, in read
    iprot.read_struct(self)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 387, in read_struct
    return read_struct(self.trans, obj, self.decode_response)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
    read_val(inbuf, f_type, f_container_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
    read_struct(inbuf, obj, decode_response)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
    read_val(inbuf, f_type, f_container_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
    read_struct(inbuf, obj, decode_response)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
    read_val(inbuf, f_type, f_container_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 256, in read_val
    result.append(read_val(inbuf, v_type, v_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
    read_struct(inbuf, obj, decode_response)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
    read_val(inbuf, f_type, f_container_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
    read_struct(inbuf, obj, decode_response)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
    read_val(inbuf, f_type, f_container_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 256, in read_val
    result.append(read_val(inbuf, v_type, v_spec, decode_response))
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 230, in read_val
    byte_payload = inbuf.read(sz)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 173, in read
    self._read_frame()
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 177, in _read_frame
    header = self._trans_read_all(4)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 198, in _trans_read_all
    return read_all(sz)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/transport/socket.py", line 110, in read
    buff = self.sock.recv(sz)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/ssl.py", line 1037, in recv
    return self.read(buflen)
  File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/ssl.py", line 913, in read
    return self._sslobj.read(len)

count queries work.
I've tried on various hosts and with multiple impala versions (the most recent one being:
impalad version 3.4.0-SNAPSHOT RELEASE (build 134517e42b7b6085e758195465f956f431e0e575)
A minimal pip freeze:

bitarray==1.6.3
impyla==0.16.3
ply==3.11
sasl==0.2.1
six==1.15.0
thrift==0.13.0
thrift-sasl==0.4.2
thriftpy2==0.4.14

, after doing:

yum install python3 python3-pip python3-devel
pip3 install impyla thrift-sasl --user