Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-17872

Dtests failing intermittently on Jolokia agent

    XMLWordPrintableJSON

Details

    Description

      Some apparently unrealeted Python dtests fail with an output of the form:

      Error Message
subprocess.CalledProcessError: Command '('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandra/cassandra/cassandra-dtest/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', 'start', '706')' returned non-zero exit status 1.
Stacktrace
self = <auth_test.TestAuthRoles object at 0x7fc6cb4313a0>

(...)

    
        mbean = make_mbean('auth', type='RolesCache')
>       with JolokiaAgent(self.cluster.nodelist()[0]) as jmx:

auth_test.py:1888: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tools/jmxutils.py:309: in __enter__
    self.start()
tools/jmxutils.py:187: in start
    subprocess.check_output(args, stderr=subprocess.STDOUT)
/usr/lib/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandr...t/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', ...),)
kwargs = {'stderr': -2, 'stdout': -1}
process = <subprocess.Popen object at 0x7fc6c9afb910>
stdout = b"Couldn't start agent for PID 706\nPossible reason could be that port '8778' is already occupied.\nPlease check the standard output of the target process for a detailed error message.\n"
stderr = None, retcode = 1

(...)

            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandra/cassandra/cassandra-dtest/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', 'start', '706')' returned non-zero exit status 1.

/usr/lib/python3.8/subprocess.py:516: CalledProcessError

      Here is a of bunch hits in different tests across multiple branches:

      Note the common with JolokiaAgent(self.cluster.nodelist()[0]) and "Possible reason could be that port '8778' is already occupied." parts.

      So far, the issue doesn't seem to reproduce on 3.0.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-17300 Test Failure: dtest-offheap.auth_test.TestNetworkAuth.test_revoked_dc_access

          • Normal -
          • Open

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-17685 Test failure: transient_replication_test.py::TestTransientReplicationRepairStreamEntireSSTable::test_optimized_primary_range_repair

          • Normal -
          • Open

          Activity

            People

              brandon.williams Brandon Williams
              adelapena Andres de la Peña
              Brandon Williams
              Andres de la Peña
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: