Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10051

impala-shell exits with ValueError with WITH clauses

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 4.0
    • Fix Version/s: Impala 4.0
    • Component/s: Clients
    • Labels:
      None
    • Epic Color:
      ghx-label-1

      Description

      Some strings can cause shlex to throw an exception in WITH clauses, for example in a regexp_replace. This should be handled more gracefully and correctly.

      Working query (impala-shell forwards the query for analysis):

      impala-shell.sh -q 'with select regexp_replace(column_name, "[a-zA-Z]", "+ ");'
      

      While same query fails with ValueError when empty spaces are removed from the arguments of the regexp_replace:

      tmate@tmate-box:~/Projects/Impala$ impala-shell.sh -q 'with select regexp_replace(column_name,"[a-zA-Z]","+ ");'
      Starting Impala Shell with no authentication using Python 2.7.16
      Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
      Opened TCP connection to localhost:21000
      Connected to localhost:21000
      Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build b29cb4ca82a4f05ea7dc0eadc330a64fbe685ef0)
      Traceback (most recent call last):
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1973, in <module>
          impala_shell_main()
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1927, in impala_shell_main
          if execute_queries_non_interactive_mode(options, query_options):
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1731, in execute_queries_non_interactive_mode
          shell.execute_query_list(queries))
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1564, in execute_query_list
          if self.onecmd(q) is CmdStatus.ERROR:
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 675, in onecmd
          return func(arg)
        File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1276, in do_with
          tokens = shlex.split(strip_comments(query.lstrip()), posix=False)
        File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 279, in split
          return list(lex)
        File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 269, in next
          token = self.get_token()
        File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 96, in get_token
          raw = self.read_token()
        File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 172, in read_token
          raise ValueError, "No closing quotation"
      ValueError: No closing quotation
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tmate Tamas Mate
                Reporter:
                tmate Tamas Mate
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: