Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-8896

'ZooKeeperMasterContenderDetectorTest.NonRetryableFrrors' is flaky

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • flaky

    Description

      This was a test failure on macOS with SSL enabled. Not sure yet if other systems might be affected as well:

      [ RUN      ] ZooKeeperMasterContenderDetectorTest.NonRetryableFrrors
      I0509 01:36:35.181434 2992141120 zookeeper_test_server.cpp:156] Started ZooKeeperTestServer on port 58450
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@753: Client environment:zookeeper.version=zookeeper C client 3.4.8
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@757: Client environment:host.name=Jenkinss-Mac-mini.local
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@764: Client environment:os.name=Darwin
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@765: Client environment:os.arch=17.4.0
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@766: Client environment:os.version=Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@774: Client environment:user.name=jenkins
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@782: Client environment:user.home=/Users/jenkins
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@log_env@794: Client environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/build
      2018-05-09 01:36:35,181:44641(0x700009f15000):ZOO_INFO@zookeeper_init@827: Initiating client connection, host=127.0.0.1:58450 sessionTimeout=10000 watcher=0x1148b6680 sessionId=0 sessionPasswd=<null> context=0x7fe697de7590 flags=0
      2018-05-09 01:36:35,182:44641(0x70000aa42000):ZOO_INFO@check_events@1764: initiated connection to server [127.0.0.1:58450]
      2018-05-09 01:36:35,185:44641(0x70000aa42000):ZOO_INFO@check_events@1811: session establishment complete on server [127.0.0.1:58450], sessionId=0x163440b82ec0000, negotiated timeout=10000
      I0509 01:36:35.186167 167882752 group.cpp:341] Group process (zookeeper-group(14)@10.0.49.4:57595) connected to ZooKeeper
      I0509 01:36:35.186213 167882752 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (1, 0, 0)
      I0509 01:36:35.186226 167882752 group.cpp:395] Authenticating with ZooKeeper using digest
      2018-05-09 01:36:38,534:44641(0x70000aa42000):ZOO_INFO@auth_completion_func@1327: Authentication scheme digest succeeded
      I0509 01:36:38.534493 167882752 group.cpp:419] Trying to create path '/mesos' in ZooKeeper
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@753: Client environment:zookeeper.version=zookeeper C client 3.4.8
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@757: Client environment:host.name=Jenkinss-Mac-mini.local
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@764: Client environment:os.name=Darwin
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@765: Client environment:os.arch=17.4.0
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@766: Client environment:os.version=Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@774: Client environment:user.name=jenkins
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@782: Client environment:user.home=/Users/jenkins
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@log_env@794: Client environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/build
      2018-05-09 01:36:38,540:44641(0x70000a121000):ZOO_INFO@zookeeper_init@827: Initiating client connection, host=127.0.0.1:58450 sessionTimeout=10000 watcher=0x1148b6680 sessionId=0 sessionPasswd=<null> context=0x7fe6999c1fe0 flags=0
      I0509 01:36:38.540652 166273024 contender.cpp:152] Joining the ZK group
      2018-05-09 01:36:38,540:44641(0x70000b463000):ZOO_INFO@check_events@1764: initiated connection to server [127.0.0.1:58450]
      2018-05-09 01:36:38,542:44641(0x70000b463000):ZOO_INFO@check_events@1811: session establishment complete on server [127.0.0.1:58450], sessionId=0x163440b82ec0001, negotiated timeout=10000
      I0509 01:36:38.542425 168955904 group.cpp:341] Group process (zookeeper-group(15)@10.0.49.4:57595) connected to ZooKeeper
      I0509 01:36:38.542466 168955904 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (1, 0, 0)
      I0509 01:36:38.542480 168955904 group.cpp:395] Authenticating with ZooKeeper using digest
      2018-05-09 01:36:50,559:44641(0x70000aa42000):ZOO_WARN@zookeeper_interest@1597: Exceeded deadline by 8687ms
      2018-05-09 01:36:50,559:44641(0x70000aa42000):ZOO_ERROR@handle_socket_error_msg@1702: Socket [127.0.0.1:58450] zk retcode=-7, errno=60(Operation timed out): connection to 127.0.0.1:58450 timed out (exceeded timeout by 5353ms)
      2018-05-09 01:36:50,559:44641(0x70000aa42000):ZOO_WARN@zookeeper_interest@1597: Exceeded deadline by 8687ms
      I0509 01:36:50.559657 167882752 group.cpp:452] Lost connection to ZooKeeper, attempting to reconnect ...
      2018-05-09 01:36:50,560:44641(0x70000b463000):ZOO_WARN@zookeeper_interest@1597: Exceeded deadline by 8687ms
      2018-05-09 01:36:50,560:44641(0x70000b463000):ZOO_ERROR@handle_socket_error_msg@1702: Socket [127.0.0.1:58450] zk retcode=-7, errno=60(Operation timed out): connection to 127.0.0.1:58450 timed out (exceeded timeout by 5352ms)
      2018-05-09 01:36:50,560:44641(0x70000b463000):ZOO_WARN@zookeeper_interest@1597: Exceeded deadline by 8687ms
      I0509 01:36:50.560917 168955904 group.cpp:452] Lost connection to ZooKeeper, attempting to reconnect ...
      ../../src/tests/master_contender_detector_tests.cpp:426: Failure
      Failed to wait 15secs for contender.contend()
      2018-05-09 01:36:53,551:44641(0x7fffb2587340):ZOO_INFO@zookeeper_close@2579: Freeing zookeeper resources for sessionId=0x163440b82ec0001
      
      2018-05-09 01:36:53,551:44641(0x7fffb2587340):ZOO_INFO@zookeeper_close@2579: Freeing zookeeper resources for sessionId=0x163440b82ec0000
      
      I0509 01:36:53.551445 2992141120 zookeeper_test_server.cpp:116] Shutting down ZooKeeperTestServer on port 58450
      [  FAILED  ] ZooKeeperMasterContenderDetectorTest.NonRetryableFrrors (18373 ms)
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            nfnt Jan Schlicht
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: