Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11251

TestImpalaShellInteractive.test_unicode_input fails on Ubuntu 20

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 4.1.0
    • Impala 4.2.0
    • Infrastructure
    • None
    • ghx-label-4

    Description

      Running TestImpalaShellInteractive.test_unicode_input() on Ubuntu results in this failure:

       

      shell/test_shell_interactive.py:377: in test_unicode_input
          child_proc.expect(PROMPT_REGEX)
      ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1451: in expect
          timeout, searchwindowsize)
      ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1466: in expect_list
          timeout, searchwindowsize)
      ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1554: in expect_loop
          raise EOF(str(err) + '\n' + str(self))
      E   EOF: End Of File (EOF). Exception style platform.
      E   <pexpect.spawn object at 0x7fe089a9e890>
      E   version: 3.3
      E   command: /home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell
      E   args: ['/home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell', '--protocol=beeswax', '-ilocalhost:21000']
      E   searcher: <pexpect.searcher_re object at 0x7fe089a9ec10>
      E   buffer (last 100 chars): ''
      E   before (last 100 chars): ' (required by /home/impdev/.python-eggs/sasl-0.2.1-py2.7-linux-x86_64.egg-tmp/sasl/saslwrapper.so)\r\n'
      E   after: <class 'pexpect.EOF'>
      E   match: None
      E   match_index: None
      E   exitstatus: None
      E   flag_eof: True
      E   pid: 1109550
      E   child_fd: 23
      E   closed: False
      E   timeout: 30
      E   delimiter: <class 'pexpect.EOF'>
      E   logfile: None
      E   logfile_read: None
      E   logfile_send: None
      E   maxread: 2000
      E   ignorecase: False
      E   searchwindowsize: None
      E   delaybeforesend: 0.05
      E   delayafterclose: 0.1
      E   delayafterterminate: 0.1

      What seems to be happening is that impala-shell is built against the system compiler / system libstdc++ (GCC 9). If we then run it with LD_LIBRARY_PATH set to point to toolchain compiler / libstdc++, then it won't find the symbols it needs to run. The solution is to use shell/util.py's spawn_shell function, which cleans up the environment for the pexpect:

       

       

      def build_shell_env(env=None):
        """ Construct the environment for the shell to run in based on 'env', or the current
        process's environment if env is None."""
        if not env: env = os.environ
        # Don't inherit PYTHONPATH or LD_LIBRARY_PATH - the shell launch script must set
        # these to include dependencies. Copy 'env' to avoid mutating argument or os.environ.
        env = dict(env)
        if "PYTHONPATH" in env:
          del env["PYTHONPATH"]
        if "LD_LIBRARY_PATH" in env:
          del env["LD_LIBRARY_PATH"]
        return env
      ...
      def spawn_shell(shell_cmd):
        """Spawn a shell process with the provided command line. Returns the Pexpect object."""
        return pexpect.spawn(shell_cmd[0], shell_cmd[1:], env=build_shell_env())

       

       

      Attachments

        Issue Links

          Activity

            People

              joemcdonnell Joe McDonnell
              joemcdonnell Joe McDonnell
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: