[IMPALA-11755] Impala-shell ldap_password_cmd fails on Python 3.8

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.2.0
• Fix Version/s: Impala 4.3.0
• Component/s: Clients
• Labels: python3
• Target Version: Impala 4.3.0
• Epic Color: ghx-label-9
• Assignee: Michael Smith
• Reporter: Michael Smith
• Created: 28/Nov/22 18:46
• Updated: 12/Apr/23 17:42
• Resolved: 29/Nov/22 22:41

Description

Python 2 - Works

(impala-shell-py2) ➜  ~ python --version
Python 2.7.18
(impala-shell-py2) ➜  ~ impala-shell --protocol='hs2-http' --ssl -i 'coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443' -u clevesque -l --ldap_password_cmd='echo -n password'
Starting Impala Shell with LDAP-based authentication using Python 2.7.18
SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change)
Warning: --connect_timeout_ms is currently ignored with HTTP transport.
Opened TCP connection to coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443
Connected to coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443
Server version: impalad version 4.0.0.2022.0.10.1-8 RELEASE (build 6ce64f72f0795f7331c93f1d1679a6426c33592d)
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v4.1.1-RELEASE (1d7b631) built on Mon Oct 17 14:08:01 PDT 2022)You can change the Impala daemon that you're connected to by using the CONNECT
command.To see how Impala will plan to run your query without actually executing
it, use the EXPLAIN command. You can change the level of detail in the EXPLAIN
output by setting the EXPLAIN_LEVEL query option.
***********************************************************************************
[coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443] default> 
[coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443] default> Goodbye clevesque 

Python 3.8

(python38-impala-shell) ➜  ~ python --version
Python 3.8.15
(python38-impala-shell) ➜  ~ 
(python38-impala-shell) ➜  ~ impala-shell --protocol='hs2-http' --ssl -i 'coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443' -u clevesque -l --ldap_password_cmd='echo -n password'
Starting Impala Shell with LDAP-based authentication using Python 3.8.15
SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change)
Warning: --connect_timeout_ms is currently ignored with HTTP transport.
Opened TCP connection to coordinator-impala-default.apps.ecs.sme-clev.athens.cloudera.com:443
Caught exception HTTP code 401: Unauthorized, type=<class 'shell_exceptions.HttpError'> in OpenSession. Num remaining tries: 3
Caught exception HTTP code 401: Unauthorized, type=<class 'shell_exceptions.HttpError'> in OpenSession. Num remaining tries: 2
Caught exception HTTP code 401: Unauthorized, type=<class 'shell_exceptions.HttpError'> in OpenSession. Num remaining tries: 1
Caught exception HTTP code 401: Unauthorized, type=<class 'shell_exceptions.HttpError'> in OpenSession. Num remaining tries: 0
Traceback (most recent call last):
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_shell.py", line 1006, in _connect
    self.server_version, self.webserver_address = self.imp_client.connect()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_client.py", line 184, in connect
    self._open_session()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_client.py", line 677, in _open_session
    resp = self._do_hs2_rpc(OpenSession, retry_on_error=True)
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_client.py", line 1021, in _do_hs2_rpc
    return rpc()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_client.py", line 673, in OpenSession
    return self.imp_service.OpenSession(open_session_req)
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/TCLIService/TCLIService.py", line 188, in OpenSession
    self.send_OpenSession(req)
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/TCLIService/TCLIService.py", line 197, in send_OpenSession
    self._oprot.trans.flush()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/thrift/transport/TTransport.py", line 178, in flush
    self.__trans.flush()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/ImpalaHttpClient.py", line 275, in flush
    raise HttpError(self.code, self.message, body, self.headers)
shell_exceptions.HttpError: HTTP code 401: UnauthorizedDuring handling of the above exception, another exception occurred:Traceback (most recent call last):
  File "/home/manishm/.virtualenvs/python38-impala-shell/bin/impala-shell", line 8, in <module>
    sys.exit(impala_shell_main())
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_shell.py", line 2111, in impala_shell_main
    with ImpalaShell(options, query_options) as shell:
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_shell.py", line 294, in __init__
    self.do_connect(options.impalad)
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_shell.py", line 952, in do_connect
    self._connect()
  File "/home/manishm/.virtualenvs/python38-impala-shell/lib/python3.8/site-packages/impala_shell/impala_shell.py", line 1046, in _connect
    self.ldap_password.endswith('\n'):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
(python38-impala-shell) ➜  ~ 

Can we please test and fix for all supported Python 3 versions.