Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10523

Impala-shell crash in printing error messages that contain UTF-8 characters

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 4.0
    • Fix Version/s: Impala 4.0
    • Component/s: Clients
    • Labels:
      None
    • Epic Link:
    • Epic Color:
      ghx-label-5

      Description

      Encounter a crash in impala-shell when playing around with a query:

      [localhost:21050] default> select cast(now() as string format 'yyyy年MM月dd日');
      Query: select cast(now() as string format 'yyyy年MM月dd日')
      Query submitted at: 2021-02-20 16:40:09 (Coordinator: http://quanlong-OptiPlex-BJ:25000)
      Query progress can be monitored at: http://quanlong-OptiPlex-BJ:25000/query_plan?query_id=6c4d64dec01254bc:d54107fd00000000
      Traceback (most recent call last):
        File "/home/quanlong/workspace/Impala/shell/impala_shell.py", line 2070, in <module>
          impala_shell_main()
        File "/home/quanlong/workspace/Impala/shell/impala_shell.py", line 2035, in impala_shell_main
          shell.cmdloop(intro)
        File "/home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/cmd.py", line 142, in cmdloop
          stop = self.onecmd(line)
        File "/home/quanlong/workspace/Impala/shell/impala_shell.py", line 697, in onecmd
          return func(arg)
        File "/home/quanlong/workspace/Impala/shell/impala_shell.py", line 1123, in do_select
          return self._execute_stmt(query_str, print_web_link=True)
        File "/home/quanlong/workspace/Impala/shell/impala_shell.py", line 1320, in _execute_stmt
          print(e, file=sys.stderr)
      UnicodeEncodeError: 'ascii' codec can't encode character u'\u5e74' in position 44: ordinal not in range(128)
      

      The crash point is in shell/impala_shell.py:1320

      1316     except QueryStateException as e:
      1317       # an exception occurred while executing the query
      1318       if self.last_query_handle is not None:
      1319         self.imp_client.close_query(self.last_query_handle)
      1320       print(e, file=sys.stderr)
      

      Definition of QueryStateException in shell/shell_exceptions.py:

       28 class QueryStateException(Exception):
       29   def __init__(self, value=""):
       30     self.value = value
       31 
       32   def __str__(self):
       33     return self.value
      

      After IMPALA-9489, 'value' of QueryStateException is in unicode type when using Python2, because we follow the "unicode sandwich" manner - "bytes on the outside, unicode on the inside, encode/decode at the edges". We should encode it to str using 'utf-8' encoding, instead of letting Python do this implicitly and fail.

        Attachments

          Activity

            People

            • Assignee:
              stigahuang Quanlong Huang
              Reporter:
              stigahuang Quanlong Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: