Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9209

TestRPCException.test_end_data_stream_error is flaky

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.4.0
    • None
    • None
    • ghx-label-9

    Description

      custom_cluster.test_rpc_exception.TestRPCException.test_end_data_stream_error (from pytest)

      Error Message

      assert (not 'Debug Action: IMPALA_SERVICE_POOL:FAIL@0.5' or 8 == 0)  +  where 8 = <bound method TestRPCException._get_num_fails of <test_rpc_exception.TestRPCException object at 0x698c790>>(<ImpaladProcess PID: 5846 (/data/jenkins/workspace/impala-cdh6.x-exhaustive/re..._SERVICE_POOL:127.0.0.1:27002:EndDataStream:FAIL@0.5 --default_query_options=)>)  +    where <bound method TestRPCException._get_num_fails of <test_rpc_exception.TestRPCException object at 0x698c790>> = <test_rpc_exception.TestRPCException object at 0x698c790>._get_num_fails
      

      Stacktrace

      custom_cluster/test_rpc_exception.py:123: in test_end_data_stream_error
          self.execute_test_query("Debug Action: IMPALA_SERVICE_POOL:FAIL@0.5")
      custom_cluster/test_rpc_exception.py:75: in execute_test_query
          assert not exception_string or self._get_num_fails(impalad) == 0
      E   assert (not 'Debug Action: IMPALA_SERVICE_POOL:FAIL@0.5' or 8 == 0)
      E    +  where 8 = <bound method TestRPCException._get_num_fails of <test_rpc_exception.TestRPCException object at 0x698c790>>(<ImpaladProcess PID: 5846 (/data/jenkins/workspace/impala-cdh6.x-exhaustive/re..._SERVICE_POOL:127.0.0.1:27002:EndDataStream:FAIL@0.5 --default_query_options=)>)
      E    +    where <bound method TestRPCException._get_num_fails of <test_rpc_exception.TestRPCException object at 0x698c790>> = <test_rpc_exception.TestRPCException object at 0x698c790>._get_num_fails
      

      Standard Error

      23:14:02 MainThread: Found 0 impalad/0 statestored/0 catalogd process(es)
      23:14:02 MainThread: Starting State Store logging to /data/jenkins/workspace/impala-cdh6.x-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
      23:14:02 MainThread: Starting Catalog Service logging to /data/jenkins/workspace/impala-cdh6.x-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
      23:14:02 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-cdh6.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
      23:14:02 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-cdh6.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
      23:14:02 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-cdh6.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
      23:14:05 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:05 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:05 MainThread: Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25000
      23:14:05 MainThread: Waiting for num_known_live_backends=3. Current value: 0
      23:14:06 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:06 MainThread: Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25000
      23:14:06 MainThread: Waiting for num_known_live_backends=3. Current value: 0
      23:14:07 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:07 MainThread: Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25000
      23:14:07 MainThread: num_known_live_backends has reached value: 3
      23:14:07 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:07 MainThread: Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25001
      23:14:07 MainThread: num_known_live_backends has reached value: 3
      23:14:08 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
      23:14:08 MainThread: Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25002
      23:14:08 MainThread: num_known_live_backends has reached value: 3
      23:14:09 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3 executors).
      DEBUG:impala_cluster:Found 3 impalad/1 statestored/1 catalogd process(es)
      INFO:impala_service:Getting metric: statestore.live-backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25010
      INFO:impala_service:Metric 'statestore.live-backends' has reached desired value: 4
      DEBUG:impala_service:Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25000
      INFO:impala_service:num_known_live_backends has reached value: 3
      DEBUG:impala_service:Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25001
      INFO:impala_service:num_known_live_backends has reached value: 3
      DEBUG:impala_service:Getting num_known_live_backends from impala-ec2-centos74-m5-4xlarge-ondemand-116e.vpc.cloudera.com:25002
      INFO:impala_service:num_known_live_backends has reached value: 3
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            twmarshall Thomas Tauber-Marshall
            stakiar Sahil Takiar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment