Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15012

Make tools/bin/token-generator py2/3 compatible

    XMLWordPrintableJSON

    Details

      Description

      While running tests for the 2.2.14 candidate I noticed that the TestTokenGenerator dtests are broken. I reproduced locally as well, the issue appears to be that we're running a python2 script with python3 (since dtests run python3 now).

      Should be a quick fix to make the token generator script py2/3 compatible.

      Example local run:

      pytest --cassandra-dir=/home/josephl/pg/cassandra_2 -k TestTokenGenerator                                                                                                                                       1 ↵
      =============================================================================================================== test session starts ================================================================================================================
      platform linux -- Python 3.6.7, pytest-3.6.4, py-1.7.0, pluggy-0.7.1
      rootdir: /home/josephl/pg/cassandra-dtest, inifile: pytest.ini
      plugins: timeout-1.3.3, flaky-3.4.0
      timeout: 900.0s
      timeout method: signal
      timeout func_only: False
      collected 1028 items / 1025 deselected                                                                                                                                                                                                             
      
      token_generator_test.py FFF                                                                                                                                                                                                                  [100%]
      
      ===================================================================================================================== FAILURES =====================================================================================================================
      _________________________________________________________________________________________________ TestTokenGenerator.test_multi_dc_tokens_default __________________________________________________________________________________________________
      
      self = <token_generator_test.TestTokenGenerator object at 0x7f5188c7cdd8>
      
          def test_multi_dc_tokens_default(self):
      >       self._multi_dc_tokens()
      
      token_generator_test.py:167: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      token_generator_test.py:153: in _multi_dc_tokens
          generated_tokens = self.call_token_generator(self.cluster.get_install_dir(), random, dc_nodes)
      token_generator_test.py:43: in call_token_generator
          token_gen_output = subprocess.check_output(args)
      /usr/lib/python3.6/subprocess.py:336: in check_output
          **kwargs).stdout
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      input = None, timeout = None, check = True, popenargs = (['/home/josephl/pg/cassandra_2/tools/bin/token-generator', '3', '5'],), kwargs = {'stdout': -1}, process = <subprocess.Popen object at 0x7f5188b80208>, stdout = b'', stderr = None
      retcode = 1
      
          def run(*popenargs, input=None, timeout=None, check=False, **kwargs):
              """Run command with arguments and return a CompletedProcess instance.
          
              The returned instance will have attributes args, returncode, stdout and
              stderr. By default, stdout and stderr are not captured, and those attributes
              will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
          
              If check is True and the exit code was non-zero, it raises a
              CalledProcessError. The CalledProcessError object will have the return code
              in the returncode attribute, and output & stderr attributes if those streams
              were captured.
          
              If timeout is given, and the process takes too long, a TimeoutExpired
              exception will be raised.
          
              There is an optional argument "input", allowing you to
              pass a string to the subprocess's stdin.  If you use this argument
              you may not also use the Popen constructor's "stdin" argument, as
              it will be used internally.
          
              The other arguments are the same as for the Popen constructor.
          
              If universal_newlines=True is passed, the "input" argument must be a
              string and stdout/stderr in the returned object will be strings rather than
              bytes.
              """
              if input is not None:
                  if 'stdin' in kwargs:
                      raise ValueError('stdin and input arguments may not both be used.')
                  kwargs['stdin'] = PIPE
          
              with Popen(*popenargs, **kwargs) as process:
                  try:
                      stdout, stderr = process.communicate(input, timeout=timeout)
                  except TimeoutExpired:
                      process.kill()
                      stdout, stderr = process.communicate()
                      raise TimeoutExpired(process.args, timeout, output=stdout,
                                           stderr=stderr)
                  except:
                      process.kill()
                      process.wait()
                      raise
                  retcode = process.poll()
                  if check and retcode:
                      raise CalledProcessError(retcode, process.args,
      >                                        output=stdout, stderr=stderr)
      E               subprocess.CalledProcessError: Command '['/home/josephl/pg/cassandra_2/tools/bin/token-generator', '3', '5']' returned non-zero exit status 1.
      
      /usr/lib/python3.6/subprocess.py:418: CalledProcessError
      -------------------------------------------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------------------------------------------
      10:03:31,457 ccm DEBUG Log-watching thread starting.
      ---------------------------------------------------------------------------------------------------------------- Captured log setup ----------------------------------------------------------------------------------------------------------------
      10:03:31,425 conftest INFO Starting execution of test_multi_dc_tokens_default at 2019-02-05 10:03:31.425612
      10:03:31,426 dtest_setup INFO cluster ccm directory: /tmp/dtest-89rd87ne
      --------------------------------------------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------------------------------------------
        File "/home/josephl/pg/cassandra_2/tools/bin/token-generator", line 160
          print "%sDC #%d:" % (indentstr, dcnum + 1)
                          ^
      SyntaxError: invalid syntax
      ------------------------------------------------------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------------------------------------------------------
      10:03:31,708 ccm DEBUG Log-watching thread exiting.
      ...
      ...
      ...
      

        Attachments

        1. test-output.png
          43 kB
          Joey Lynch
        2. --graph output.png
          11 kB
          Joey Lynch

          Activity

            People

            • Assignee:
              jolynch Joey Lynch
              Reporter:
              jolynch Joey Lynch
              Authors:
              Joey Lynch
              Reviewers:
              Stefan Podkowinski
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: