Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10541

impyla fails to fetchall() in python3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Clients
    • Labels:
      None
    • Epic Color:
      ghx-label-4

      Description

      select * queries fail instantly, with:

      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 536, in fetchall
          return list(self)
        File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 582, in __next__
          self._buffer = self._last_operation.fetch(self.description,
        File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 187, in description
          schema = self._last_operation.get_result_schema()
        File "/home/balazsj/.local/lib/python3.6/site-packages/impala/hiveserver2.py", line 1295, in get_result_schema
          entry = column.typeDesc.types[0].primitiveEntry
      AttributeError: 'NoneType' object has no attribute 'types'
      

      Selecting specific columns hangs indefinitely. The stack trace is always:

       File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 535, in fetchall
          return list(self)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 583, in __next__
          convert_types=self.convert_types)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1242, in fetch
          resp = self._rpc('FetchResults', req)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 992, in _rpc
          response = self._execute(func_name, request)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1009, in _execute
          return func(request)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 219, in _req
          return self._recv(_api)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 238, in _recv
          result.read(self._iprot)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/thrift.py", line 160, in read
          iprot.read_struct(self)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 387, in read_struct
          return read_struct(self.trans, obj, self.decode_response)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
          read_val(inbuf, f_type, f_container_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
          read_struct(inbuf, obj, decode_response)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
          read_val(inbuf, f_type, f_container_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
          read_struct(inbuf, obj, decode_response)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
          read_val(inbuf, f_type, f_container_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 256, in read_val
          result.append(read_val(inbuf, v_type, v_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
          read_struct(inbuf, obj, decode_response)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
          read_val(inbuf, f_type, f_container_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 289, in read_val
          read_struct(inbuf, obj, decode_response)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 316, in read_struct
          read_val(inbuf, f_type, f_container_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 256, in read_val
          result.append(read_val(inbuf, v_type, v_spec, decode_response))
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/protocol/binary.py", line 230, in read_val
          byte_payload = inbuf.read(sz)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 173, in read
          self._read_frame()
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 177, in _read_frame
          header = self._trans_read_all(4)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 198, in _trans_read_all
          return read_all(sz)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/site-packages/thriftpy2/transport/socket.py", line 110, in read
          buff = self.sock.recv(sz)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/ssl.py", line 1037, in recv
          return self.read(buflen)
        File "/Users/balazsjeszenszky/.pyenv/versions/3.7.3/lib/python3.7/ssl.py", line 913, in read
          return self._sslobj.read(len)
      

      count queries work.
      I've tried on various hosts and with multiple impala versions (the most recent one being:
      impalad version 3.4.0-SNAPSHOT RELEASE (build 134517e42b7b6085e758195465f956f431e0e575)
      A minimal pip freeze:

      bitarray==1.6.3
      impyla==0.16.3
      ply==3.11
      sasl==0.2.1
      six==1.15.0
      thrift==0.13.0
      thrift-sasl==0.4.2
      thriftpy2==0.4.14
      

      , after doing:

      yum install python3 python3-pip python3-devel
      pip3 install impyla thrift-sasl --user
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jeszyb Balazs Jeszenszky
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: