Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-14320

dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes

    XMLWordPrintableJSON

    Details

      Description

      JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its _query function tries to use json.loads (which only accepts string input) on a bytes object.

          def _query(self, body, verbose=True):
              request_data = json.dumps(body).encode("utf-8")
              url = 'http://%s:8778/jolokia/' % (self.node.network_interfaces['binary'][0],)
              req = urllib.request.Request(url)
              response = urllib.request.urlopen(req, data=request_data, timeout=10.0)
              if response.code != 200:
                  raise Exception("Failed to query Jolokia agent; HTTP response code: %d; response: %s" % (response.code, response.readlines()))
      
              raw_response = response.readline() # response is http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes when read
              response = json.loads(raw_response) # this raises a TypeError now
              if response['status'] != 200:
                  stacktrace = response.get('stacktrace')
                  if stacktrace and verbose:
                      print("Stacktrace from Jolokia error follows:")
                      for line in stacktrace.splitlines():
                          print(line)
                  raise Exception("Jolokia agent returned non-200 status: %s" % (response,))
              return response

      This can be seen clearly by running the deprecated repair tests (repair_tests/deprecated_repair_test.py). They all fail right now because of this TypeError.

      This is a side effect of the migration to Python 3, which makes bytes objects fundamentally different from strings. This will also happen anytime we try to json.loads data returned from stdout or stderr piped from subprocess. I need to take a closer look at offline_tools_test.py and cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.

      We can fix this issue by decoding bytes objects to strings before calling json.loads(). For example, in the above:

              response = json.loads(raw_response.decode(encoding='utf-8'))

      I have a fix for the JolokiaAgent problem - I'll submit a pull request to cassandra-dtest once I have this issue number to reference.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ptbannister Patrick Bannister
                Reporter:
                ptbannister Patrick Bannister
                Authors:
                Patrick Bannister
                Reviewers:
                Marcus Eriksson
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: